17 Nov
|
Kadel Labs
|
India
Position: Databricks Architect Number of Positions: 1
Experience:
10–12+ Years (with strong hands-on Databricks architecture experience)
Location:
Udaipur, Jaipur, Bhopal, Bangalore Employment Type: Full-Time Role Overview We are looking for an experienced Databricks Architect to lead the design, architecture, and implementation of enterprise-scale data platforms using the Databricks Lakehouse ecosystem. The ideal candidate should have solid expertise in Unity Catalog, Delta Lake, data governance, Lakehouse Monitoring, dimensional modeling, and platform cost optimization. The candidate will be responsible for establishing scalable, secure, and cost-efficient data architectures while ensuring compliance with governance requirements, particularly for sensitive healthcare (PHI) and financial datasets. Strong stakeholder management and executive communication skills are essential. Key Responsibilities
· Design and implement scalable enterprise data architectures using the Databricks Lakehouse platform.
· Define and implement data governance frameworks using Unity Catalog, including access controls, data lineage, metadata management, and auditing.
· Architect and optimize data storage and processing solutions using Delta Lake, including performance tuning and data reliability.
· Leverage Databricks System Tables for platform observability, usage analytics, workload monitoring, and operational insights.
· Implement and manage Lakehouse Monitoring to track data quality, data drift, anomalies, and pipeline health.
· Design enterprise data models, including dimensional modeling, Star Schema, Snowflake Schema, and fact/dimension structures.
· Establish FinOps practices for Databricks, including compute optimization, workload management,
cluster utilization, and cost monitoring.
· Define data security and compliance strategies for sensitive datasets, including PHI and financial data.
· Establish best practices for data access management, encryption, auditability, and regulatory compliance.
· Collaborate with engineering teams, business stakeholders, and senior leadership to translate business requirements into technical architecture.
· Provide architectural guidance, conduct design reviews, and mentor data engineering teams.
· Present architecture decisions, technical recommendations, platform performance, and cost optimization strategies to executive stakeholders. Requirements Required Technical Skills Skill Area Required Expertise Databricks Strong enterprise-level Databricks architecture experience Unity Catalog Governance, access controls, lineage, metadata management Delta Lake Delta tables, optimization, performance tuning, reliability System Tables Platform monitoring, usage analytics, audit and cost insights Lakehouse Monitoring Data quality, anomaly detection, drift monitoring Data Modeling Dimensional modeling, Star Schema, Snowflake Schema FinOps Cost optimization, compute management, usage monitoring Data Governance Data security, privacy, compliance,
access policies Sensitive Data PHI and financial data governance Cloud Platforms Azure / AWS / GCP with Databricks Required Experience & Competencies
· Extensive experience in enterprise data architecture and data engineering.
· Proven experience designing and implementing Databricks-based Lakehouse solutions.
· Strong understanding of enterprise data governance, security, and compliance frameworks.
· Experience working with healthcare data, including PHI, and/or financial datasets.
· Ability to define architectural standards and guide large-scale data platform implementations.
· Strong communication skills with the ability to interact with C-level executives and business stakeholders.
· Ability to communicate complex technical concepts in business-oriented language. Good to Have
· Databricks certifications such as Databricks Certified Data Engineer Professional or Databricks Certified Data Architect.
· Experience with Azure Data Factory, AWS Glue, or other data integration platforms.
· Knowledge of HIPAA and other data privacy and regulatory requirements.
· Experience with Power BI, Tableau, or other enterprise BI platforms.
· Experience leading enterprise-wide data modernization and cloud migration initiatives. Key Deliverables 1. Enterprise Databricks Lakehouse architecture and implementation roadmap. 2. Unity Catalog-based enterprise data governance framework. 3. Optimized Delta Lake architecture and performance standards. 4. System Tables and Lakehouse Monitoring implementation. 5. Dimensional data modeling standards. 6. Databricks FinOps and cost optimization framework. 7. Security and compliance architecture for PHI and financial data. 8. Architecture documentation and executive-level technical presentations.
📌 Databricks Architect (India)
🏢 Kadel Labs
📍 India