Job Purpose
To effectively design, develop, and manage data solutions using ETL technologies such as Azure Databricks (ADB) , Azure Data Factory (ADF) and SQL
Duties and Responsibilities
KEY ROLES / PRINCIPAL ACCOUNTABILITIES
Data Engineering & Platform Development
• Build scalable pipelines using Azure Databricks (PySpark, SQL)
• Develop and orchestrate ETL workflows using Azure Data Factory
• Work with Delta Lake architecture (Bronze–Silver–Gold layers)
• Enable real-time and batch data processing pipelines
• Enable data exposure via APIs for BI and downstream systems
CI/CD & DevOps
• Implement CI/CD pipelines for data and AI workflows
• Automate deployments across environments (Dev, QA, Prod)
• Ensure version control and reproducibility
Database Proficiency:
Strong knowledge of SQL and experience with relational databases like SQL Server, MySQL, etc.

________________________________________
?
KEY RESPONSIBILITIES
• Translate business requirements into technical solutions in collaboration with the PMO team.
• Own end-to-end delivery of data projects, ensuring on-time execution and adherence to quality standards.
• Design technical architecture and guide development efforts for enhancements and recent projects.
• Develop and maintain robust ETL pipelines and data integration modules across systems.
• Ensure high data quality, data anomaly resolution of critical process issues.
• Monitor and resolve performance bottlenecks in data workflows and programs.
• Establish best practices, standard operating procedures, and drive their implementation across teams.




• Act as a liaison with business users and product managers to support daily data needs and strategic initiatives.
• Coordinate with internal and external development teams to troubleshoot and resolve issues efficiently.
• Manage workload through effective planning, prioritization, and progress tracking.
Key Decisions / Dimensions
KEY DECISIONS / DIMENSIONS
• Define semantic layer design and metric definitions
• Prioritize data vs AI optimization trade-offs
• Handle production issues with RCA and long-term fixes
• Drive architectural decisions for lakehouse + Data integration
Major Challenges
MAJOR CHALLENGES
• Ensuring Data Delivery within TAT
• Driving adoption of GenAI-based BI over traditional dashboards
• Balancing performance, cost, and scalability
• Managing dependencies across data engineering, AI, and business teams
Required Qualifications and Experience
REQUIRED SKILLS & EXPERIENCE
Must Have
• Azure Databricks – PySpark, SQL, Delta Lake
• Strong experience in Semantic Modeling & Metrics Layer design
• Hands-on with Databricks workflows
• Pyspark (Pandas, PySpark, FastAPI)
• Azure Data Factory (ADF) for ETL pipelines
• Strong SQL and data modeling skills
Good to Have
• Cosmos DB / MongoDB (NoSQL concepts)
• Azure Data Explorer (KQL)
________________________________________
DATA STACK (MANDATORY FOR SCREENING)
SNo Data Platform / Concepts Associated Technologies
1 Databricks Lakehouse PySpark, SQL, Delta Lake
2 AI for BI Databricks Genie, Genie Rooms, Instructions, Agents
4 ETL & Orchestration Azure Data Factory
5 Programming Pyspark
6 Cloud Platform Azure (Preferred)
10 DevOps CI/CD Pipelines, Git

📌 Senior Data Engineer (India)
🏢 Bajaj Finance
📍 India