Lead Data Engineer

We are hiring a Lead Data Engineer to design, develop, and optimize scalable cloud data platforms and modern ETL/ELT pipelines using Microsoft Fabric, Databricks, or Snowflake.

The role involves building and maintaining data lakehouse and data warehouse solutions, developing high-performance data pipelines, enabling analytics and reporting, and contributing to data engineering best practices.

Solid hands-on experience in PySpark, SQL, Python/Scala, and cloud data platforms is required.

What You Will Do

Following are the high-level responsibilities that you will play, but are not limited to:

- Design, develop, and maintain scalable data pipelines, data models, and ETL/ELT processes.- Build and optimize data lake, lakehouse, and data warehouse solutions using Microsoft Fabric, Databricks, or Snowflake.- Develop data processing solutions using PySpark, Python, Scala, and SQL.- Work with large and complex datasets to improve data processing performance, reliability, and scalability.- Develop and maintain batch and real-time data pipelines as required.- Implement data transformation, cleansing, validation, and integration processes.- Work with Delta Lake, Spark, Snowflake, Microsoft Fabric, and other modern data engineering technologies.- Support business analytics and self-service reporting through Power BI and other visualization platforms.- Collaborate with data scientists, analysts, BI developers, and business stakeholders to deliver reliable data solutions.- Implement data quality, governance, security, and monitoring practices across data pipelines.- Troubleshoot and resolve data pipeline,



performance, and production issues.- Contribute to the development of coding standards, reusable frameworks, and data engineering best practices.- Participate in code reviews and ensure high-quality, maintainable, and scalable solutions.- Implement CI/CD practices for data engineering workflows and deployments.- Stay updated with emerging data engineering technologies and contribute to continuous improvement of the data ecosystem.

Required Qualifications

- Bachelor's/Master’s degree in Computer Science, Information Technology, Engineering, or a related field.- 6–10 years of experience in data engineering, data platform development, or related areas.

Strong hands-on experience in one or more of the following:

- Microsoft Fabric – Data Factory, Lakehouse, Data Warehouse- Databricks – Spark, Delta Lake, PySpark, MLflow- Snowflake – Data Warehousing, Snowpipe, Performance Optimization- Power BI – Data Modeling, DAX, Report Development- Strong proficiency in SQL.- Strong programming skills in Python or Scala.- Hands-on experience with PySpark/Apache Spark and large-scale data processing.- Experience working with Azure, AWS, or GCP cloud data services.- Strong understanding of data modeling, ETL/ELT concepts, data warehousing, and data lake/lakehouse architectures.- Understanding of data governance, data quality, security, and access control.- Experience with CI/CD, Git, and DevOps practices** for data engineering workflows.- Strong analytical and problem-solving skills.- Ability to work independently and collaborate effectively with cross-functional teams.

📌 Lead Data Engineer (Mumbai)
🏢 Thinkwise Consulting
📍 Mumbai