We are looking for an experienced Senior Databricks Architect with 10–15 years of IT experience to lead enterprise-scale Lakehouse architecture, platform modernization, data governance, and migration initiatives. The candidate should have solid hands-on expertise in Databricks, Apache Spark, PySpark, Delta Lake, and Unity Catalog, along with the ability to define architecture strategies, engineering standards, performance optimization, and cloud data platform roadmaps.

Key Responsibilities

Design and lead enterprise-scale Databricks Lakehouse architecture, including workspace strategy, Unity Catalog, storage, compute, and Medallion architecture.

Define architecture standards and best practices for Databricks, PySpark, Spark SQL, Delta Lake, DLT/Lakeflow, Workflows, and Auto Loader.

Lead large-scale data platform modernization and migration from legacy data warehouses, Hadoop, and ETL platforms to Databricks.

Drive Spark performance optimization, including query tuning, AQE, partitioning, caching, shuffle optimization, and Photon.

Design enterprise data governance, security, access controls, lineage, data quality, and Delta Sharing frameworks using Unity Catalog.

Establish CI/CD, DevOps, Infrastructure as Code (Terraform), automated testing, and deployment standards for Databricks platforms.

Define platform reliability, monitoring, observability, disaster recovery, and production support standards.

Drive Databricks cost optimization through cluster policies, autoscaling, compute sizing, serverless adoption, and FinOps practices.





Build AI-ready data foundations and demonstrate awareness of RAG, Vector Search, MLflow, Model Serving, and AI governance.

Lead architecture workshops, engage with business and technical stakeholders, define technology roadmaps, and mentor engineering teams.

Mandatory Skills

10–15 years of overall IT experience with significant experience in Data Engineering and Architecture.

Strong expertise in Databricks Lakehouse Architecture and Apache Spark.

Hands-on experience with PySpark, Spark SQL, Delta Lake, and Medallion Architecture.

Strong knowledge of Unity Catalog, Data Governance, Security, and Data Lineage.

Experience designing enterprise-scale cloud data platforms on AWS, Azure, or GCP.

Expertise in data migration, modernization, performance tuning, and platform optimization.

Experience with CI/CD, DevOps, Terraform/IaC, and production-grade data platform engineering.

Strong architecture leadership, stakeholder management, solution design, and technical mentoring skills.

Good to Have

Experience with Lakeflow Declarative Pipelines / Delta Live Tables (DLT), Auto Loader, and Databricks Workflows.

Knowledge of Structured Streaming, CDC, SCD, and Delta Sharing.

Exposure to Databricks Asset Bundles, system tables, Lakehouse Federation, and serverless compute.

Awareness of MLflow, Vector Search, RAG, and AI/ML platform integration.

Experience In Enterprise FinOps, Chargeback/showback, And Cloud Cost Governance.

Skills: sql,tuning,databricks,migration,spark sql,apache spark,pyspark,lakehouse,unity,aws,ci/cd

📌 Senior Databricks Architect (Bengaluru)
🏢 Zorba AI
📍 Bengaluru