Role Summary

Design, develop, and maintain scalable data engineering solutions on the Azure platform using Azure Databricks, Azure Data Factory, ADLS Gen2, and related Azure data services. Build enterprise-grade data pipelines, transformation frameworks, and Lakehouse architectures.

Key Responsibilities

- Design and implement end-to-end data pipelines using Azure Databricks and Azure Data Factory.

- Develop scalable ETL/ELT solutions using PySpark and Spark SQL.

- Build and maintain Lakehouse architectures using Bronze, Silver and Gold layers.

- Develop Delta Lake-based data models and optimize data processing performance.

- Integrate data from ERP, CRM, databases, APIs, IoT, and external sources.

- Implement data quality checks, monitoring, and error-handling frameworks.

- Create and maintain CI/CD pipelines using Azure DevOps.

- Configure and manage Unity Catalog, data governance, and access controls.





- Work with Power BI and analytics teams to deliver curated data marts and semantic models.

- Collaborate with business stakeholders, architects, and data scientists to support reporting and AI initiatives.

- Troubleshoot production issues and optimize pipeline performance.

- Prepare technical documentation, deployment guides, and operational runbooks.

Required Technical Skills

- Azure Databricks

- PySpark and Spark SQL

- Azure Data Factory (ADF)

- Azure Data Lake Storage Gen2 (ADLS)

- Delta Lake

- SQL Server and SQL scripting

- Azure DevOps and CI/CD

- Git/Version Control

- Data Warehousing and ETL Concepts

- Performance Tuning and OptimizationRole & responsibilities

Preferred candidate profile

📌 Azure Databricks Engineers (Ahmedabad)
🏢 EY
📍 Ahmedabad