Role & responsibilities
- Data Modeling & Analytics Engineering
- Design, develop, and maintain scalable dimensional data models, including Star and Snowflake schemas
- Build curated data products that support reporting, analytics, AI, and machine learning use cases
- Define and standardize business logic, KPIs, metrics, and calculations across enterprise domains
- Create reusable datasets, semantic models, and data assets to improve consistency and reduce duplication
- Collaborate with business stakeholders to translate requirements into analytical and reporting solutions
- Databricks Platform Engineering
- Design and implement modern data architectures leveraging Databricks Lakehouse principles
- Develop and maintain Bronze, Silver, and Gold data layers
- Build scalable ETL/ELT pipelines using Spark, PySpark, SQL, and Databricks Workflows
- Optimize Delta Lake implementations using partitioning, Z-Ordering, data skipping, Change Data Feed (CDF)
- Delta optimization techniques
- Implement CI/CD pipelines and DevOps best practices for data engineering and analytics solutions
- Context Layer & Semantic Layer Development
- Design and manage enterprise semantic and context layers to provide a single source of truth for reporting and analytics
- Standardize business metrics, dimensions, hierarchies, and definitions across reporting platforms
- Enable self-service analytics through Power BI, Tableau, and AI-driven applications
- Manage semantic models and governance processes to ensure reporting consistency
- Establish metric certification, lineage tracking, and data governance standards
- Performance Optimization & Cost Management
- Analyze query execution plans and identify performance bottlenecks
- Optimize Spark workloads for scalability, reliability, and cost efficiency
- Implement cluster sizing, autoscaling, caching, broadcast joins, and Adaptive Query Execution (AQE) strategies
- Tune Delta Lake tables through file compaction, partition optimization, and storage management
- Monitor warehouse and cluster utilization to improve performance and control cloud expenses
- Enhance dashboard and reporting performance through optimized data models and query design
- Data Governance & Quality Management
- Develop and implement data quality frameworks, validation processes, and automated monitoring
- Ensure compliance with enterprise security, governance, and regulatory requirements
- Support metadata management, data cataloging, and lineage initiatives
- Establish monitoring and alerting for data pipelines, processing failures, and data anomalies
- Partner with governance teams to drive data stewardship and certification processes
- Technical Leadership & Collaboration
- Collaborate with data scientists, analysts, engineers, product owners, and business leaders
- Mentor and guide junior analytics engineers and data engineers
- Conduct architecture reviews and recommend technology best practices
- Drive adoption of enterprise data standards, frameworks, and governance processes
- Lead technical design discussions and influence data platform strategy and roadmap
- Comply with all applicable Company policies, procedures, and business directives, changes including those relating to work location, team assignments, work schedules, and versatile work arrangements
Preferred candidate profile
- Master's degree in Computer Science, Data Science, Analytics, or a related discipline
- Databricks certifications (Data Engineer Associate, Professional, or equivalent)
- Experience with Databricks Unity Catalog and enterprise data governance implementations
- Experience supporting AI/ML, Generative AI, or advanced analytics use cases
- Experience designing enterprise semantic models and metric layers
- Healthcare, insurance, or highly regulated industry experience
- Demonstrated experience leading technical teams, mentoring engineers, and driving enterprise data strategy
- Knowledge of data catalog and governance tools such as Purview, Collibra, or Alation
Key Competencies:
- Data Architecture & Modeling
- Analytics Engineering
- Databricks & Delta Lake
- Spark/PySpark Development
- Data Governance & Quality
- Performance Optimization
- Semantic Layer Design
- Cloud Data Platforms
- Technical Leadership
- Stakeholder Management
📌 Data Engineering Lead (Noida)
🏢 Optum
📍 Noida