Role & responsibilities

Job Summary

We are seeking an experienced Azure Data Engineer with over 8 years of experience in designing, developing, and implementing scalable data engineering solutions on Microsoft Azure. The ideal candidate should have strong expertise in Azure Databricks, Azure Data Factory, SQL, and cloud-based ETL/ELT pipelines. The role involves building high-performance data platforms, optimizing data processing, and collaborating with cross-functional teams to deliver enterprise-grade analytics solutions.

Key Responsibilities

- Design, develop, and maintain scalable data pipelines using Azure Data Factory (ADF).
- Develop and optimize ETL/ELT solutions using Azure Databricks and PySpark.
- Create reusable notebooks and workflows for large-scale data processing.
- Design and implement Delta Lake architecture and optimize Spark workloads.
- Develop complex SQL queries, stored procedures, views, and performance tuning techniques.
- Integrate data from multiple structured and unstructured data sources.
- Build data ingestion frameworks using Azure Data Lake Storage (ADLS Gen2).
- Perform data validation, reconciliation, and quality checks.
- Collaborate with business analysts, architects, and data scientists to understand business requirements.




- Implement monitoring, logging, and error-handling mechanisms for data pipelines.
- Optimize Azure resources for performance, scalability, and cost efficiency.
- Participate in code reviews, design discussions, and production support activities.
- Follow DevOps best practices using Git and CI/CD pipelines.
- Ensure compliance with data security, governance, and organizational standards.
- Mentor junior team members and provide technical guidance.

Required Skills

- 8+ years of experience in Data Engineering and ETL development.
- Solid experience with Azure Data Factory (ADF).
- Strong experience with Azure Databricks.
- Hands-on experience with PySpark and Apache Spark.
- Advanced SQL development and query optimization.
- Experience with Azure Data Lake Storage (ADLS Gen2).
- Knowledge of Delta Lake architecture.
- Experience in designing cloud-based data pipelines.
- Strong understanding of data warehousing concepts, including Star and Snowflake schemas.
- Experience with Git and Azure DevOps.
- Familiarity with Agile/Scrum methodologies.
- Excellent analytical, troubleshooting, and communication skills.

Preferred candidate profile

📌 Azure Data Engineer (Pune)
🏢 Advance Career Solutions
📍 Pune