11 Aug
|
statsby.ai
|
Pune
We are looking for an experienced Data Engineer (Lead) with strong expertise in
Databricks to design, develop, and maintain scalable data pipelines and data
processing solutions.
KEY RESPONSIBILITIES
Design, develop, and maintain scalable data pipelines using Databricks.
Develop data processing solutions using PySpark and Python.
Work with Apache Spark for large-scale data processing.
Build and optimize ETL/ELT pipelines.
Work with Delta Lake for data storage and processing.
Develop and manage workflows using Databricks Workflows.
Perform data transformation, cleansing, and validation.
Optimize Spark jobs and improve pipeline performance.
Collaborate with data engineers, analysts, and other technical teams.
Troubleshoot data pipeline issues and ensure data quality.
REQUIRED SKILLS
3–5 years of experience in Data Engineering.
Strong hands-on experience with Databricks.
Good knowledge of PySpark and Python.
Strong understanding of Apache Spark.
Experience with Delta Lake and data lake architecture.
Valuable knowledge of SQL.
Experience in building ETL/ELT data pipelines.
Understanding of cloud platforms such as Azure, AWS, or GCP.
Experience with Git and CI/CD is an added advantage.
GOOD TO HAVE
Experience with Azure Databricks.
Knowledge of Azure Data Factory, ADLS, or similar cloud data services.
Experience with DBT, Snowflake, or other modern data platforms.
Databricks certification is a plus.
WHAT WE OFFER
Prospect to work on real-world Data Engineering and AI projects.
Exposure to modern data technologies and cloud platforms.
Team-oriented and learning-focused work setting.
Opportunity to work with a growing technology team.
📌 Databricks Lead Pune
🏢 statsby.ai
📍 Pune