Roles & Responsibilities

- Design, build, and maintain scalable data solutions on Microsoft Azure, Microsoft Fabric, and Databricks to support business data needs.
- Develop and manage data pipelines for ingesting, transforming, and integrating data from multiple sources using Spark-based processing frameworks.
- Data modeling for Data Warehousing and Lakehouse projects to build scalable and reliable analytical platforms.
- Implement data security, governance, lineage, and privacy controls across Azure data platforms.
- Monitor, troubleshoot, and optimize data workflows to ensure reliability, performance, and cost efficiency.
- Optimize data storage and processing solutions using Azure, Fabric, Databricks, Spark, and Delta Lake best practices.
- Design and support batch and real-time data processing solutions.

Role Summary An Azure Data Engineer is responsible for designing, building, and maintaining scalable data frameworks on Azure to support enterprise data requirements. The role involves developing and managing end-to-end data pipelines, integrating data from multiple sources, and ensuring reliable data ingestion, transformation, and processing using Microsoft Fabric, Databricks, PySpark, Spark SQL, and Azure data services.

The Azure Data

Engineer also implements data security, governance, and privacy controls while optimizing data platforms for performance, scalability, and cost efficiency across modern Azure data platforms.

Work Experience

Minimum 6+ years of Azure Data Engineering experience with hands-on expertise in Microsoft Fabric, Databricks, Spark-based data processing

Preferred Technical & Functional Skills

- Provide technical leadership to data engineering teams, including solution design, implementation guidance, and effective client demos and presentations with strong English communication skills.
- Design, develop, and deploy scalable ETL/ELT pipelines using Azure Data Factory,



Synapse Analytics, Microsoft Fabric, Databricks, Azure Functions, and Notebooks.
- Strong hands-on experience with Microsoft Fabric and Databricks, including Lakehouse Architecture, Medallion Architecture, OneLake, Delta Lake, Fabric Data Factory, and Fabric Notebooks.

• Strong proficiency in PySpark, Spark SQL, Delta Lake, and distributed data processing frameworks.

- Experience building metadata-driven ingestion, orchestration, and data quality frameworks.
- Hands-on experience implementing SCD Type 1 & Type 2, CDC, and incremental processing frameworks.
- Experience in Spark performance tuning, Delta Lake optimization, workload management, and cost-efficient data platform design.

• Solid understanding of Unity Catalog, data governance, lineage, access management, and security controls.

• Proficiency in Data Modeling, Dimensional Modeling, Lakehouse, and Enterprise Data Warehouse design .

- Strong knowledge of Azure RBAC and IAM, with experience implementing security and compliance standards.

Good to Have

- Experience handling streaming and near real-time datasets using Spark Structured Streaming, Event Hub, Kafka, or similar technologies.
- Exposure to Microsoft Copilot and Generative AI fundamentals, with the ability to identify and apply industry-relevant AI use cases to data engineering workflows.
- Relevant Microsoft role-based certifications (DP-600, DP-700, DP-203, DP-900, AI-102, AI-900); Power BI (PL-300) certification is a plus. Databricks certifications are desirable.
- Proficient in Power BI for building dashboards, defining KPIs, and delivering actionable business insights.
- Experience with Azure DevOps, Git, mentoring junior team members, and managing data engineering projects in consulting environments.
- Strong alignment with Microsoft's vision and roadmap, with an eagerness to adopt emerging tools across the Data & AI ecosystem.

Location

Bangalore / Pune / Hyderabad

📌 KDN India_Consulting_Azure Data Engineer (Bengaluru)
🏢 KPMG
📍 Bengaluru