30 Sep
|
ClusterNeuron
|
Ahmedabad
30 Sep
ClusterNeuron
Ahmedabad
DATABRICKS ARCHITECT WITH AI
Experience: 15+ Years
Location: Ahmedabad
Work Mode: On-site
Employment Type: FTE / Contract
Contract Duration: 1 Year
Contract Budget: ₹2.2 LPM
Role Overview
We are looking for an experienced Databricks Data Architect to lead the design and implementation of enterprise-scale data platforms using Databricks Lakehouse Architecture.
Key Responsibilities
- Design and implement enterprise-scale data platforms using Databricks Lakehouse Architecture.
- Define end-to-end data architecture covering ingestion, transformation, storage, governance, and consumption.
- Establish architecture standards, coding guidelines, reusable design patterns, and best practices.
- Lead technical decisions related to scalability, performance, reliability, and security.
- Design and build scalable ETL/ELT pipelines using PySpark on Databricks.
- Architect Bronze, Silver, and Gold data transformation frameworks.
- Design and manage Delta Lake storage architecture.
- Implement Databricks Workflows / Lakeflow pipelines.
- Define data quality, validation, reconciliation, and monitoring frameworks.
- Oversee CI/CD and DevOps practices.
- Design enterprise data models using dimensional modeling techniques.
- Develop Star Schemas, Fact Tables, Dimension Tables, and conformed dimensions.
- Define business metrics, KPI layers, semantic models, and curated Gold datasets.
- Provide technical leadership and mentorship to data engineers and developers.
- Conduct architecture and code reviews.
- Troubleshoot complex technical issues and guide teams in solution design.
- Support solution proposals, estimation,
technical governance, and architectural roadmap activities.
Mandatory Skills
- Data Architecture
- Databricks Lakehouse Architecture
- Databricks Workflows / Lakeflow
- Delta Lake
- Unity Catalog
- Databricks SQL
- PySpark
- Spark SQL
- Python
- SQL
- ETL / ELT
- Dimensional Data Modeling
- Star Schema
- Fact & Dimension Modeling
- Slowly Changing Dimensions (SCD)
- Metric Views
- Git and CI/CD
- Performance Optimization
- Data Quality Frameworks
- Batch and Incremental Processing
Cloud & Integration
- Azure Databricks
- ADLS Gen2
- Azure Data Factory
- Event-based and batch ingestion
- Data governance
- Metadata management
- Data security
Good to Have
- Data warehouse modernization
- Legacy EDW migration to Databricks
- Data Mesh
- Medallion Architecture
- Power BI, Tableau, or Looker
- Data cataloging and governance
Candidate Profile
- 15+ years of relevant experience in data architecture and enterprise data platforms.
- Strong hands-on experience with Databricks and modern data engineering.
- Experience designing scalable and high-performance data platforms.
- Robust expertise in data modeling, Lakehouse architecture, and cloud data solutions.
- Ability to define target-state architecture and technical roadmaps.
- Experience reviewing technical designs and code.
- Strong communication and stakeholder management skills.
- Strong leadership and mentoring capabilities.
Mandatory Skills: Data Architecture, Databricks Pay: Up to ₹200,000.00 per month
Work Location: In person
📌 DATABRICKS ARCHITECT WITH AI (Ahmedabad)
🏢 ClusterNeuron
📍 Ahmedabad