Job Title : Lead Data Engineer / Data Engineer (Python)

Experience: 8+ Years

Location: Chennai, Pune, Bengaluru, Hyderabad

Contract: 6 Months

Notice Period : Immediate Joiners

We are looking for an experienced Lead Data Engineer / Data Engineer with robust expertise in Python, Scala, Spark, and large-scale data engineering to join our team.

Key Responsibilities
- Lead the design, development, deployment, and maintenance of scalable data pipelines.
- Define data engineering architecture, technical strategies, and best practices.
- Build optimized batch and streaming data pipelines in cloud environments.
- Design and maintain ETL, Data Lake, and database pipeline architectures.
- Develop solutions for data validation, analysis, monitoring, and alerting.
- Work with large datasets and write highly optimized SQL queries.
- Ensure data quality, reliability, governance, security, and lifecycle management.




- Review code and drive performance, scalability, and engineering best practices.
- Collaborate with Product, Business, QA, and Engineering teams.
- Mentor engineers through technical discussions, code reviews, and knowledge sharing.
- Troubleshoot complex technical issues and drive continuous improvement.

Must-Have Skills

Python

Scala

Apache Spark

Microsoft Fabric

Data Engineering & Data Pipelines

Data Lake / Delta Lake / Hive

Batch & Streaming Processing

Advanced SQL

MySQL / Snowflake

RDBMS / OLAP

ETL & Data Architecture

Cloud Computing

Data Governance & Security

Distributed Systems / Big Data

Observability, Monitoring & Alerting

Agile methodologies

Team Leadership & Mentoring

📌 Lead Data Engineer (Pune)
🏢 RapidBrains
📍 Pune