Our client is looking for a skilled Data Engineer – Databricks with experience in data engineering and solid hands-on expertise in Azure Databricks, PySpark, SQL, and Delta Lake. The candidate will be responsible for developing, optimizing, and maintaining scalable data pipelines and implementing reliable data processing solutions.

Key Responsibilities

- Design, develop, and maintain scalable data pipelines using Azure Databricks, PySpark, and SQL.
- Build and manage data ingestion and transformation processes using Delta Lake and Delta Tables.
- Implement Auto Loader (cloudFiles), schema inference, and schema evolution.
- Develop and manage Spark Declarative Pipelines (SDP/DLT).
- Configure and manage Databricks Workflows, Jobs, Tasks, and Triggers.
- Implement data governance, security, and access management using Unity Catalog.
- Design dimensional data models, data marts, and Slowly Changing Dimensions (SCD).
- Develop and optimize Databricks SQL Dashboards and work with AI/BI, Genie, or similar BI tools.
- Monitor and troubleshoot data pipelines to ensure performance, reliability, and scalability.

Requirements





- 6–8 years of overall experience in Data Engineering.
- Strong hands-on experience with Azure Databricks, PySpark, and SQL.
- Expertise in Delta Lake and Delta Tables.
- Practical experience with Auto Loader (cloudFiles), schema inference, and schema evolution.
- Hands-on experience with Spark Declarative Pipelines (SDP/DLT).
- Good understanding of Databricks Workflows, Jobs, Tasks, and Triggers.
- Strong knowledge of Unity Catalog, data governance, and access management.
- Experience with dimensional modelling, data marts, and SCD.
- Familiarity with Databricks SQL Dashboards, AI/BI, and Genie.
- Working knowledge of cloud platforms such as Azure, AWS, or GCP.

Qualifications

- Bachelor's degree in Computer Science, Information Technology, Engineering, or a related discipline.
- Strong understanding of data engineering concepts, ETL/ELT processes, and data pipeline architecture.
- Excellent analytical, problem-solving, and troubleshooting skills.
- Ability to work independently and collaborate effectively with cross-functional teams.

📌 Data Engineer – Databricks (Pune)
🏢 Talentmatics
📍 Pune