Azure/Databricks Senior Data Engineer

We are currently hiring for a Lead Data Engineer for one of our clients (a leading Software Services/Product Company) based out of Mumbai.

We are looking for experienced Data Engineering professionals who are passionate about building modern, scalable, and cloud-native data platforms.

The ideal candidate should have solid expertise in Azure Databricks, Apache Spark, Azure Data Services, and enterprise-scale data engineering.

This role requires both hands-on technical leadership and the ability to mentor engineering teams while delivering high-quality data solutions.

This role offers above-market compensation along with standard benefits for the right candidate.

Key Responsibilities :

- Design, build, and maintain scalable end-to-end data pipelines for data ingestion, transformation, and serving.
- Develop and optimize large-scale data processing solutions using Azure Databricks and Apache Spark.
- Design and implement Delta Lake architectures following Bronze, Silver, and Gold data layers.
- Build cloud-native data solutions using Azure Data Lake Storage Gen2 (ADLS Gen2), Azure Data Factory, Azure Key Vault, Event Grid, and API Management.
- Lead and execute enterprise data migration initiatives, including on-premises migrations, cross-workspace migrations, Unity Catalog migrations, and historical data loads.
- Implement CI/CD pipelines and automation for data engineering workflows and infrastructure.
- Develop Infrastructure as Code (IaC) solutions using Terraform and Databricks Asset Bundles.
- Implement data governance, security, and access management using Unity Catalog, RBAC, and Service Principals.
- Monitor production environments, troubleshoot incidents, and ensure platform reliability and performance.
- Lead technical discussions, mentor engineering teams, conduct code reviews,



and drive engineering best practices across projects.

Required Skills/Primary Skills :

- 10+ years of experience in Data Engineering and Big Data technologies.
- Minimum 3+ years of hands-on experience with Azure Databricks and Apache Spark.
- Strong experience building end-to-end enterprise data pipelines.
- Deep understanding of Delta Lake Architecture including Bronze, Silver, and Gold data layers.
- Strong hands-on experience with Azure Data Lake Storage Gen2 (ADLS Gen2).
- Experience with Azure Data Factory (ADF) for orchestration and pipeline development.
- Hands-on experience with Azure Key Vault, Event Grid, and Azure API Management.
- Experience delivering large-scale cloud-based data platforms on Microsoft Azure.
- Strong experience with enterprise data migration projects, including historical data migration and platform modernization.
- Experience implementing CI/CD pipelines for data engineering workloads.
- Hands-on expertise with Terraform and Databricks Asset Bundles.
- Strong understanding of data governance using Unity Catalog, RBAC, and Service Principals.
- Experience supporting production environments, monitoring data platforms, and resolving production incidents.

Additional Skills :

- Experience leading and mentoring Data Engineering teams.
- Strong understanding of software engineering best practices, code reviews, and technical governance.
- Experience designing highly available, scalable, and secure cloud data platforms.
- Strong problem-solving and troubleshooting skills in distributed data processing environments.
- Experience working in Agile development environments.
- Excellent communication and stakeholder management skills.
- Ability to drive architecture decisions and establish engineering best practices across teams.

Notice Period :

- Immediate Joiners Preferred.

📌 Senior Data Engineer - Azure Databricks (India)
🏢 Build Up Rise Up UG
📍 India