Job Purpose To effectively design, develop, and manage data solutions using ETL technologies such as Azure Databricks (ADB), Azure Data Factory (ADF), and SQL along with leading 3 to 5 members of developers.

Duties and ResponsibilitiesKEY RESPONSIBILITIES

- Lead the end-to-end design, development, and delivery of scalable data engineering solutions using Azure Databricks, ADF, PySpark, SQL, and Delta Lake.
- Convert business requirements into robust technical designs, architecture documents, data models, and implementation plans.
- Own technical delivery of data integration, ETL, lakehouse, semantic layer, and AI/BI enablement initiatives.
- Guide and mentor data engineers on coding standards, design best practices, performance optimization, and reusable framework development.
- Review technical designs, code, pipelines, and deployment plans to ensure quality, scalability, maintainability, and compliance.
- Drive architecture decisions for batch and near-real-time data pipelines across Bronze, Silver, and Gold layers.
- Ensure data quality, reconciliation, anomaly detection, and timely resolution of production issues through effective RCA and permanent fixes.
- Optimize data pipelines, Databricks jobs, SQL queries, and storage usage to improve performance and reduce cost.
- Implement CI/CD practices, version control, automated deployments, and environment management across Dev, QA, and Production.
- Collaborate with PMO, business stakeholders, BI teams, InfoSec, DevOps, and external partners for smooth project execution.
- Establish SOPs, engineering standards, reusable components, monitoring frameworks, and documentation practices.
- Track delivery progress, manage technical dependencies, prioritize work, and ensure timely closure of project milestones.
- Support adoption of modern data platforms, semantic modeling, metrics layer design, and GenAI/BI capabilities.




- Ensure compliance with data governance, security, access control, audit, and enterprise data management standards.
- Act as the technical escalation point for critical issues, complex solutioning, and cross-team dependency resolution.

Key Decisions / DimensionsKEY DECISIONS / DIMENSIONS

- Define semantic layer design and metric definitions.
- Prioritize data vs AI optimization trade-offs.
- Handle production issues with RCA and long-term fixes.
- Drive architectural decisions for lakehouse Data integration.

Major ChallengesMAJOR CHALLENGES

- Ensuring Data Delivery within TAT.
- Driving adoption of GenAI-based BI over traditional dashboards.
- Balancing performance, cost, and scalability.
- Managing dependencies across data engineering, AI, and business teams.

Required Qualifications and ExperienceREQUIRED SKILLS & EXPERIENCE Must Have

- Azure Databricks - PySpark, SQL, Delta Lake.
- Strong experience in Semantic Modeling & Metrics Layer design.
- Hands-on with Databricks workflows.
- Pyspark (Pandas, PySpark, FastAPI).
- Azure Data Factory (ADF) for ETL pipelines.
- Strong SQL and data modeling skills.

Positive to Have

- Cosmos DB / MongoDB (NoSQL concepts).
- Azure Data Explorer (KQL).

DATA STACK (MANDATORY FOR SCREENING) | SNo | Data Platform / Concepts | Associated Technologies | |-----|--------------------------|-----------------------------------------| | 1 | Databricks Lakehouse | PySpark, SQL, Delta Lake | | 2 | AI for BI | Databricks Genie, Genie Rooms, Instructions, Agents | | 4 | ETL & Orchestration | Azure Data Factory | | 5 | Programming | Pyspark | | 6 | Cloud Platform | Azure (Preferred) | | 10 | DevOps | CI/CD Pipelines, Git |

LocationsBASIC SECTION

- Job Level: GB05
- Job Title: Associate Delivery Manager - Data Engineer - IT, Data Technologies, Data Technology
- Job Location Country: India
- State: MAHARASHTRA
- Region: West
- City: Pune
- Location Name: Pune Corporate Office - Mantri Tier Megapolis

📌 Associate Delivery Manager - Data Engine... (India)
🏢 Bajaj FinServ
📍 India