Lead Data Engineer - Azure:
Key Responsibilities:
- Design, develop, and maintain scalable data pipelines using Python and PySpark.
- Build and optimize ETL/ELT workflows for processing structured and unstructured data.
- Develop and manage data engineering solutions using Azure Databricks, Azure Synapse Analytics, and Azure Data Factory (ADF).
- Create high-performance SQL queries, stored procedures, and database objects for analytics and reporting.
- Design and implement modern data lake and data warehouse architectures on Azure.
- Lead technical design discussions, code reviews, and implementation of best practices.
- Collaborate with business stakeholders, data analysts, data scientists, and solution architects to gather requirements and deliver scalable solutions.
- Optimize data pipelines for performance, scalability, reliability, and cost efficiency.
- Implement data quality checks, monitoring, logging, and error-handling mechanisms.
- Ensure data security, governance, and compliance with organizational standards.




- Mentor junior and mid-level data engineers and foster engineering best practices.
- Support production deployments, troubleshooting, and performance tuning.

Required Skills:
- Strong programming experience in Python.
- Hands-on expertise in PySpark and distributed data processing.
- Extensive experience with ETL/ELT design and implementation.
- Strong knowledge of Azure Databricks.
- Experience with Azure Synapse Analytics.
- Hands-on experience with Azure Data Factory (ADF).
- Advanced SQL development and query optimization skills.
- Experience with Azure Data Lake Storage (ADLS Gen2).
- Knowledge of data modeling concepts (Star Schema, Snowflake Schema, Dimensional Modeling).
- Experience with Git and CI/CD pipelines.
- Robust understanding of data engineering best practices and performance optimization.

📌 Lead de (Gurugram)
🏢 Switch- Now Consulting
📍 Gurugram