Job Description

Lead Data Engineer

n

We are hiring a **Lead Data Engineer** to design, develop, and optimize scalable cloud data platforms and modern ETL/ELT pipelines using **Microsoft Fabric, Databricks, or Snowflake**.

n

The role involves building and maintaining data lakehouse and data warehouse solutions, developing high-performance data pipelines, enabling analytics and reporting, and contributing to data engineering best practices.

n

Strong hands-on experience in **PySpark, SQL, Python/Scala, and cloud data platforms** is required.

n

What You Will Do

n

Following are the high-level responsibilities that you will play, but are not limited to:

n

n
- Design, develop, and maintain scalable data pipelines, data models, and ETL/ELT processes.
n
- Build and optimize data lake, lakehouse, and data warehouse solutions using **Microsoft Fabric, Databricks, or Snowflake**.
n
- Develop data processing solutions using **PySpark, Python, Scala, and SQL**.
n
- Work with large and complex datasets to improve data processing performance, reliability, and scalability.
n
- Develop and maintain batch and real-time data pipelines as required.
n
- Implement data transformation, cleansing, validation, and integration processes.
n
- Work with **Delta Lake, Spark, Snowflake, Microsoft Fabric**, and other modern data engineering technologies.
n
- Support business analytics and self-service reporting through **Power BI** and other visualization platforms.
n
- Collaborate with data scientists, analysts, BI developers, and business stakeholders to deliver reliable data solutions.
n
- Implement data quality, governance, security, and monitoring practices across data pipelines.
n
- Troubleshoot and resolve data pipeline,



performance, and production issues.
n
- Contribute to the development of coding standards, reusable frameworks, and data engineering best practices.
n
- Participate in code reviews and ensure high-quality, maintainable, and scalable solutions.
n
- Implement CI/CD practices for data engineering workflows and deployments.
n
- Stay updated with emerging data engineering technologies and contribute to continuous improvement of the data ecosystem.
n

n

Required Qualifications

n

n
- Bachelor's/Master's degree in Computer Science, Information Technology, Engineering, or a related field.
n
- 6–10 years of experience** in data engineering, data platform development, or related areas.
n

n

Solid hands-on experience in one or more of the following:

n

n
- Microsoft Fabric** – Data Factory, Lakehouse, Data Warehouse
n
- Databricks** – Spark, Delta Lake, PySpark, MLflow
n
- Snowflake** – Data Warehousing, Snowpipe, Performance Optimization
n
- Power BI** – Data Modeling, DAX, Report Development
n
- Strong proficiency in **SQL**.
n
- Strong programming skills in **Python or Scala**.
n
- Hands-on experience with **PySpark/Apache Spark** and large-scale data processing.
n
- Experience working with **Azure, AWS, or GCP** cloud data services.
n
- Strong understanding of data modeling, ETL/ELT concepts, data warehousing, and data lake/lakehouse architectures.
n
- Understanding of data governance, data quality, security, and access control.
n
- Experience with **CI/CD, Git, and DevOps practices** for data engineering workflows.
n
- Strong analytical and problem-solving skills.
n
- Ability to work independently and collaborate effectively with cross-functional teams.
n

📌 Lead Data Engineer (Hyderabad)
🏢 Thinkwise Consulting
📍 Hyderabad