Job Title : Lead Data Engineer / Data Engineer (Python)

Experience: 8+ Years

Location: Chennai, Pune, Bengaluru, Hyderabad

Contract: 6 Months

Notice Period : Immediate Joiners

We are looking for an experienced Lead Data Engineer / Data Engineer with robust expertise in Python, Scala, Spark, and large-scale data engineering to join our team.

Key Responsibilities
Lead the design, development, deployment, and maintenance of scalable data pipelines.
Define data engineering architecture, technical strategies, and best practices.
Build optimized batch and streaming data pipelines in cloud settings.
Design and maintain ETL, Data Lake, and database pipeline architectures.
Develop solutions for data validation, analysis, monitoring, and alerting.
Work with large datasets and write highly optimized SQL queries.
Ensure data quality, reliability, governance, security, and lifecycle management.




Review code and drive performance, scalability, and engineering best practices.
Collaborate with Product, Business, QA, and Engineering teams.
Mentor engineers through technical discussions, code reviews, and knowledge sharing.
Troubleshoot complex technical issues and drive continuous improvement.

Must-Have Skills

Python

Scala

Apache Spark

Microsoft Fabric

Data Engineering & Data Pipelines

Data Lake / Delta Lake / Hive

Batch & Streaming Processing

Advanced SQL

MySQL / Snowflake

RDBMS / OLAP

ETL & Data Architecture

Cloud Computing

Data Governance & Security

Distributed Systems / Big Data

Observability, Monitoring & Alerting

Agile methodologies

Team Leadership & Mentoring

📌 Lead Data Engineer Pune (India)
🏢 RapidBrains
📍 India