Senior Azure Data Engineer

Job Description

We are looking for a highly skilled Azure Data Engineer with solid hands-on experience in Azure Databricks, PySpark, Azure Data Factory (ADF), ADLS Gen2, Delta Live Tables (DLT), and Unity Catalog to build and maintain enterprise-scale data platforms and modern Lakehouse architectures.

The ideal candidate should have experience designing scalable ETL/ELT pipelines, implementing Medallion Architecture, optimizing Spark workloads, and driving data governance through Unity Catalog.

The ideal candidate should have experience designing scalable ETL/ELT pipelines, implementing Medallion Architecture, optimizing Spark workloads, and driving data governance through Unity Catalog.

Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines using Azure Data Factory and Azure Databricks.
- Build and implement Medallion Architecture (Bronze, Silver, Gold layers).
- Develop data transformation logic using PySpark and Spark SQL.
- Build and manage Delta Live Tables (DLT) pipelines.
- Implement data governance, security,



and access controls using Unity Catalog.
- Design and manage data ingestion frameworks using ADLS Gen2 and Auto Loader.
- Optimize Databricks jobs and Spark workloads for performance and cost efficiency.
- Develop metadata-driven and reusable data pipelines.
- Implement monitoring, logging, and data quality checks.
- Work closely with business and technology teams to gather requirements and deliver data solutions.
- Support CI/CD deployment using Azure DevOps.
- Participate in production support, troubleshooting, and performance tuning activities.

Mandatory Skills
- Azure Databricks
- Azure Data Factory (ADF)
- PySpark
- SQL
- ADLS Gen2
- Delta Live Tables (DLT)
- Unity Catalog
- Delta Lake
- Medallion Architecture
- ETL / ELT Development

Good to Have
- Auto Loader
- Azure Synapse Analytics
- Microsoft Fabric
- Databricks Workflows
- Structured Streaming
- Azure Event Hub
- Kafka
- Azure DevOps

📌 Azure Data Engineer (Bengaluru)
🏢 KPMG Assurance and Consulting Services
📍 Bengaluru