Mandatory Requirements
- 8-14 years of Data Engineering experience.
- 4-8 years of hands-on Databricks experience.
- Experience modernizing legacy code into PySpark.
- Production-scale data pipeline development experience.
- Experience working in Agile environments.
Mandatory Certification-
Must Possess At Least One Of The Following
- Databricks Certified Data Engineer Associate
- Databricks Certified Data Engineer Professional
Job Summary:
We are looking for an experienced
Lead Data Engineer
to design, develop, and optimize scalable data pipelines on the
Databricks
platform. The ideal candidate will have strong expertise in
PySpark
,
Databricks
,
Delta Lake
, and modern data engineering practices while leading the modernization of legacy ETL pipelines into cloud-native data solutions.
Essential Technical Skills
- Data Engineering: Strong foundation in data engineering principles, ETL/ELT processes, and data pipeline design patterns
- PySpark: Proven hands-on experience developing data pipelines using PySpark,
including DataFrames API, Spark SQL, and performance optimization
- Databricks Platform: Practical experience with Databricks workspace, cluster management, notebooks, and job orchestration
- Workspace AI Agent: Knowledge of Databricks Workspace AI Agent capabilities and integration
- Data Modelling: Experience implementing data models including dimensional modeling, data vault, or lakehouse architectures
- Delta Lake: Understanding of Delta Lake features including ACID transactions, schema evolution, and optimization techniques
- Python: Solid Python programming skills for data processing and automation
- ]:pointer-events-auto R6Vx5W_threadScrollVars scroll-mb-[calc(var(--scroll-root-safe-area-inset-bottom,0px)+var(--thread-response-height))] scroll-mt-[calc(var(--header-height)+min(200px,max(70px,20svh)))]" dir="auto" data-turn-id="request-WEB:c31b1b9c-8b90-4850-a24f-ffb6013969b0-0" data-turn-id-container="requ
📌 Lead Data Engineer (Pune Division)
🏢 YO IT CONSULTING
📍 Pune Division
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.