Job Purpose
To design and deliver AI-powered Business Intelligence solutions by integrating Databricks Genie, Semantic Models, and Metric Views on top of a robust end-to-end data engineering stack.
The role focuses on enabling self-service analytics using natural language (GenAI), ensuring business-friendly data consumption, and building governed, scalable, and high-performance data ecosystems across batch and real-time pipelines.
Duties and Responsibilities
KEY ROLES / PRINCIPAL ACCOUNTABILITIES
AI for BI & GenAI Enablement
• Design and implement GenAI-powered BI solutions using Databricks Genie or MS Fabric
• Create and manage Data Rooms for business users with curated datasets
• Define and maintain Instructions Layer (prompt engineering for business context)
• Enable natural language to SQL/insights workflows for self-service analytics
• Drive adoption of Databricks One & other platforms as a unified analytics interface
Semantic Layer & Metrics Engineering
• Design and manage Semantic Models for business abstraction
• Develop reusable and governed Metric Views (KPIs, aggregations, business definitions)
• Ensure consistency across BI tools (Power BI / Genie)
• Align semantic layer with business glossary and data governance policies
Data Engineering & Platform Development
• Build scalable pipelines using Azure Databricks (PySpark, SQL)
• Develop and orchestrate ETL workflows using Azure Data Factory
• Work with Delta Lake architecture (Bronze–Silver–Gold layers)
• Enable real-time and batch data processing pipelines
• Enable data exposure via APIs for BI and downstream systems
CI/CD & DevOps




• Implement CI/CD pipelines for data and AI workflows
• Automate deployments across environments (Dev, QA, Prod)
• Ensure version control and reproducibility
________________________________________
?
KEY RESPONSIBILITIES
• Translate business problems into AI-driven BI solutions
• Own end-to-end delivery: ingestion ? transformation ? semantic layer ? AI consumption
• Design scalable data architecture aligned with lakehouse principles
• Ensure data quality, governance, and metric consistency
• Collaborate with business teams to onboard them onto Genie-based analytics
• Optimize performance of queries, pipelines, and AI responses
• Establish best practices for AI in BI (prompting, semantic tuning, governance)
• Drive adoption of self-service BI with minimal dependency on tech teams
Key Decisions / Dimensions
KEY DECISIONS / DIMENSIONS
• Define semantic layer design and metric definitions
• Decide GenAI prompting strategies and instruction frameworks
• Prioritize data vs AI optimization trade-offs
• Handle production issues with RCA and long-term fixes
• Drive architectural decisions for lakehouse + AI integration
Major Challenges
MAJOR CHALLENGES
• Ensuring accuracy and trust in AI-generated insights




• Driving adoption of GenAI-based BI over traditional dashboards
• Maintaining semantic consistency across multiple tools
• Balancing performance, cost, and scalability
• Managing dependencies across data engineering, AI, and business teams
Required Qualifications and Experience
REQUIRED SKILLS & EXPERIENCE
Must Have
• Azure Databricks – PySpark, SQL, Delta Lake
• Solid experience in Semantic Modeling & Metrics Layer design
• Hands-on with Databricks Genie / GenAI-based BI workflows
• Databricks One (Unified BI Experience)
• Prompt Engineering / Instruction tuning for AI systems
• Python (Pandas, PySpark, FastAPI)
• Azure Data Factory (ADF) for ETL pipelines
• Strong SQL and data modeling skills
Good to Have
• Cosmos DB / MongoDB (NoSQL concepts)
• Azure Data Explorer (KQL)
________________________________________
DATA STACK (MANDATORY FOR SCREENING)
SNo Data Platform / Concepts Associated Technologies
1 Databricks Lakehouse PySpark, SQL, Delta Lake
2 AI for BI Databricks Genie, Genie Rooms, Instructions, Agents
3 Semantic Layer Semantic Models, Metric Views
4 ETL & Orchestration Azure Data Factory
5 Programming Python, C#/.NET
6 Cloud Platform Azure (Preferred)
10 DevOps CI/CD Pipelines, Git
________________________________________
?
SOFT SKILLS
• Strong analytical and problem-solving mindset
• Ability to bridge business + data + AI
• Effective stakeholder communication
• Ownership-driven and proactive approach
• Ability to work in fast-paced, evolving AI landscape

📌 Senior Data Engineer (Pune)
🏢 Bajaj Finance
📍 Pune