8-14 years of Data Engineering experience.
4-8 years of hands-on Databricks experience.
Experience modernizing legacy code into PySpark.
Production-scale data pipeline development experience.
Experience working in Agile environments.
Mandatory Certification-
Must Possess At Least One Of The Following
Databricks Certified Data Engineer Associate
Databricks Certified Data Engineer Professional
Job Summary: We are looking for an experienced Lead Data Engineer to design, develop, and optimize scalable data pipelines on the Databricks platform. The ideal candidate will have strong expertise in PySpark, Databricks, Delta Lake, and modern data engineering practices while leading the modernization of legacy ETL pipelines into cloud-native data solutions.
Essential Technical Skills
Data Engineering: Strong foundation in data engineering principles, ETL/ELT processes, and data pipeline design patterns
PySpark: Proven hands-on experience developing data pipelines using PySpark, including DataFrames API,
Spark SQL, and performance optimization
Databricks Platform: Practical experience with Databricks workspace, cluster management, notebooks, and job orchestration
Workspace AI Agent: Knowledge of Databricks Workspace AI Agent capabilities and integration
Data Modelling: Experience implementing data models including dimensional modeling, data vault, or lakehouse architectures
Delta Lake: Understanding of Delta Lake features including ACID transactions, schema evolution, and optimization techniques
Python: Strong Python programming skills for data processing and automation
]:pointer-events-auto R6Vx5W_threadScrollVars scroll-mb-[calc(var(--scroll-root-secure-area-inset-bottom,0px)+var(--thread-response-height))] scroll-mt-[calc(var(--header-height)+min(200px,max(70px,20svh)))]" dir="auto" data-turn-id="request-WEB:c31b1b9c-8b90-4850-a24f-ffb6013969b0-0" data-turn-id-container="request-WEB:c31b1b9c-8b90-4850-a2
📌 Lead Data Engineer (Pune)
🏢 YO IT CONSULTING
📍 Pune
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.