Key Responsibilities
Design, develop, and optimize data pipelines using Databricks, PySpark, Delta Lake, and related Azure components.
Work across pipeline development, data modeling, and production code optimization, ensuring scalability and performance.
Build current pipelines from scratch as well as enhance and maintain existing ones.
Apply strong understanding of ETL design principles, data modeling concepts, and data governance standards.
Collaborate within a scrum team, taking ownership of assigned stories while independently delivering high-quality, production-ready code.
Demonstrate proficiency across PySpark, SQL, Delta Lake, Unity Catalog, and Databricks Workflows, with solid understanding of logic and data flow.
Work in Azure settings, leveraging tools like ADLS, ADF, and Synapse (as applicable).




Contribute inputs to architecture and design discussions where appropriate, while primarily focusing on hands-on development.
Troubleshoot data issues, perform root-cause analysis, and optimize performance for existing pipelines.

Requirements

Desired Skills and Experience
3–5 years of experience in data engineering, preferably with Azure Databricks.
Solid technical expertise in PySpark, SQL, and Delta Lake.
Familiarity with Unity Catalog, data governance, and DevOps practices (Git, CI/CD).
Ability to work independently as well as collaboratively within an agile delivery team.
Excellent problem-solving, debugging, and communication skills.

📌 Sr Databricks Developer Pune
🏢 ScatterPie Analytics
📍 Pune