Job Description (JD)
Lead / Senior Data Engineer
Location: Hyderabad, India
Experience: 8+ Years
Employment Type: Full-Time
Notice Period : Immediate / 15 Days.

About the Role
We are looking for a highly skilled and results-driven Lead / Senior Data Engineer with extensive experience in designing, developing, and managing enterprise-scale data platforms across Azure and AWS cloud ecosystems. The ideal candidate will bring strong expertise in Azure Databricks, Snowflake, Delta Lake, Unity Catalog, Delta Live Tables (DLT), Azure Data Factory, Python, PySpark, and SQL , combined with proven leadership experience in delivering modern data warehouse and lakehouse solutions.
This role requires a hands-on technical leader who can drive architecture decisions, build scalable and governed data platforms, mentor engineering teams, and collaborate closely with business stakeholders to deliver data-driven solutions that enable analytics, reporting, and AI/ML initiatives.

Key Responsibilities
Data Architecture & Engineering
Design, develop, and maintain scalable and high-performance ETL/ELT pipelines using Azure Data Factory, Azure Databricks, Python, PySpark, and SQL.
Build metadata-driven and parameterized data ingestion frameworks supporting both batch and real-time processing.
Architect modern Lakehouse and Data Warehouse solutions leveraging Azure and AWS cloud technologies.
Develop analytics-ready datasets and enterprise data products for business intelligence, reporting, and advanced analytics use cases.
Implement robust data quality frameworks, reconciliation processes, and validation controls.
Azure Databricks & Lakehouse Development
Design and implement Medallion Architecture (Bronze, Silver, Gold) using Delta Lake.
Build and manage Delta Live Tables (DLT) pipelines for reliable and quality-enforced data processing.
Develop PySpark and Spark SQL transformations for large-scale data workloads.
Implement Unity Catalog governance, including data lineage, auditing, metadata management, and fine-grained access controls.
Optimize Databricks workloads through partitioning strategies, caching, broadcast joins, Adaptive Query Execution (AQE),



and data-skew handling.
Snowflake Data Warehousing
Design and implement Snowflake-based data warehouse solutions.
Develop and optimize Snowflake schemas, data models, RBAC frameworks, and data ingestion processes using Snowpipe and COPY operations.
Perform SQL query tuning, warehouse sizing, performance optimization, and cost management.
Support reporting, analytics, and enterprise BI workloads through scalable and efficient data models.
Cloud & Streaming Solutions
Develop real-time and event-driven data processing solutions using Azure Event Hubs, AWS Kinesis, Azure Functions, and AWS Lambda.
Build scalable ingestion frameworks for streaming and batch data sources.
Support hybrid and multi-cloud data ecosystems across Azure and AWS platforms.
Data Modeling & Governance
Design dimensional data models using Star Schema and Snowflake Schema methodologies.
Implement Slowly Changing Dimensions (SCD Type 1, 2, and 3).
Support Data Mesh and domain-oriented data product initiatives.
Establish enterprise data governance practices, security controls, and data stewardship standards.
DevOps & Automation
Develop and maintain CI/CD pipelines using Azure DevOps and GitHub Actions.
Implement automated deployment, monitoring, testing, and release management processes.
Promote engineering best practices through code reviews, version control standards, and development frameworks.
Leadership & Stakeholder Management
Lead technical design discussions, architectural reviews, and project delivery activities.
Mentor and guide data engineering teams through technical coaching and knowledge sharing.
Collaborate with business stakeholders, architects, data scientists, and analytics teams to define and implement data solutions.




Drive technical strategy and ensure successful project execution from design through production support.

Required Qualifications
Education
Bachelor's Degree in Computer Science, Information Technology, Engineering, or a related discipline.
Experience
8+ years of experience in Data Engineering, Data Warehousing, and Cloud Data Platforms.
Proven experience leading enterprise data transformation and modernization initiatives.
Strong track record of delivering scalable data solutions in Azure cloud environments.

Technical Skills
Cloud Platforms
Microsoft Azure
Amazon Web Services (AWS)
Azure Technologies
Azure Databricks
Azure Data Factory (ADF)
Azure Synapse Analytics
Azure Data Lake Storage Gen2 (ADLS Gen2)
Azure SQL Database
Azure Event Hubs
Azure Functions
Microsoft Fabric
Databricks & Data Governance
PySpark
Spark SQL
Delta Lake
Delta Live Tables (DLT)
Unity Catalog
Databricks Workflows
Data Warehousing
Snowflake
Azure Synapse Analytics
Amazon Redshift
Programming Languages
Python
PySpark
SQL / T-SQL
Pandas
AWS Services
AWS Glue
Amazon S3
Amazon Athena
AWS Lambda
AWS Step Functions
AWS IAM
Amazon CloudWatch
Data Modeling
Medallion Architecture
Star Schema & Snowflake Schema
Data Warehousing
Data Mesh
Lakehouse Architecture
SCD Type 1, Type 2, and Type 3
DevOps & Collaboration
Azure DevOps
GitHub Actions
Git Version Control
Agile/Scrum Methodologies

Preferred Qualifications
Databricks Certification (Associate/Professional).
Snowflake Certification.
Microsoft Azure Data Engineering Certifications.
Experience supporting AI/ML and advanced analytics workloads.
Experience implementing enterprise data governance frameworks.
Exposure to Microsoft Fabric and up-to-date Lakehouse platforms.

Key Competencies
Technical Leadership
Solution Architecture
Data Engineering Best Practices
Performance Optimization
Problem Solving & Root Cause Analysis
Stakeholder Management
Team Mentoring & Coaching
Strategic Decision Making
Communication & Collaboration

📌 Senior Data Engineer (Hyderabad)
🏢 Anblicks
📍 Hyderabad