- We are seeking an experienced Databricks Data Architect to lead the design architecture and implementation of enterprisescale data platforms on Databricks
- The ideal candidate should possess strong expertise in modern data engineering practices dimensional data modeling and Databricks Lakehouse architecture
- The candidate will act as the technical owner of the data platform providing architectural guidance establishing best practices mentoring development teams and ensuring successful delivery of scalable highperformance data solutions
Key Responsibilities
- Architecture Solution Design Design and implement enterprisescale data platforms using the Databricks Lakehouse architecture
- Define endtoend data architecture including ingestion transformation storage governance and consumption layers
- Establish architectural standards framework guidelines coding best practices and reusable design patterns Lead technical decisions related to scalability performance optimization reliability and security Conduct architecture reviews and provide recommendations to project teams Data Engineering Databricks Implementation
- Design and build scalable ETLELT pipelines using PySpark on Databricks Architect and implement Bronze Silver Gold data transformation frameworks Design and manage Delta Lake based storage architecture including partitioning optimization and data lifecycle management
- Implement Databricks Lakeflow pipelines for data orchestration and automation
- Define data quality validation reconciliation and monitoring frameworks Oversee CICD and DevOps practices for data engineering projects Data Modeling Analytics Enablement Design and implement enterprise data models using Dimensional Modeling techniques
- Develop Star Schemas Fact Tables Dimension Tables and conformed dimensions
- Define business metrics KPI layers and semantic models
- Architect and implement Metric Views and curated Gold datasets for reporting and analytics
- Work closely with business stakeholders to translate requirements into scalable analytical models
- Technical Leadership
- Provide technical leadership and mentorship to data engineers and developers
- Perform code reviews and ensure adherence to architecture standards
- Troubleshoot complex technical issues and guide teams in solution design
- Support estimation solution proposals and technical governance activities
- Drive adoption of contemporary data engineering and cloudnative best practices
Required Technical Skills
- Core Databricks Skills
- Databricks Lakehouse Architecture
- Databricks Workflows Lakeflow
- Delta Lake
- Unity Catalog
- Databricks SQL
- Performance Tuning Optimization
- Data Engineering
- PySpark Spark SQL ETL ELT Design Data Pipeline Development Data Quality Frameworks Batch and Incremental Processing Data Modeling Dimensional Data Modeling Star Schema Design Fact Dimension Modeling Slowly Changing Dimensions SCD Metric View
- Design Analytical Data Modeling Cloud Integration Azure Databricks preferred ADLS Gen2 Azure Data Factory ADF Eventbased and Batch Ingestion Patterns Data Governance and Security Programming Python SQL Git CICD Frameworks
Desired Skills
- Data Warehouse Modernization experience
- Experience migrating legacy EDW solutions to Databricks Exposure to Data Mesh and Medallion Architecture
- Knowledge of Data Governance Metadata Management and Data Cataloging Experience supporting BI platforms such as Power BI Tableau or Looker Leadership Expectations
- The candidate should be capable of Owning the complete Databricks implementation lifecycle
- Defining targetstate architecture and roadmap
- Reviewing designs and code produced by team members
- Guiding developers on PySpark optimization and data modeling best practices
- Driving technical discussions with client architects and business stakeholders Ensuring scalability maintainability and longterm sustainability of the platform
- Providing architectural governance across multiple workstreams
Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.
📌 Associate Principal - Data Engineering (Mumbai)
🏢 LTM
📍 Mumbai