Databricks Data Engineering Architect

Primary Responsibilities

- Define and drive the

Databricks Data Engineering architecture and strategy

, aligned with enterprise Data & Analytics standards.

- Take end-to-end ownership of Databricks solutions from architecture and design through implementation, production deployment, support, and optimization.

- Define and enforce policy-based governance and cost controls across Databricks workloads, including Apps, Agents, compute, AI/ML resources, and

AI Gateway

.

- Establish and enforce

RBAC and least-privilege security across Unity Catalog, Databricks Apps, Agents, AI Gateway, service principals, and Dev/QA/Production environments.

- Design reusable data ingestion and transformation frameworks using

Databricks, Spark, Delta Lake, SQL, Python/PySpark, and Lakeflow

.

- Architect scalable

Lakehouse solutions on Databricks and AWS supporting batch, streaming, near-real-time, analytics, and AI use cases.

- Design and support

Agentic AI and Databricks Apps development

, including architecture, deployment, integration, security, monitoring, and production readiness.

- Establish and enforce

Data Engineering Governance and Security using Unity Catalog, including RBAC, least privilege, service principals, data access controls, lineage, auditing, and data quality.

- Define engineering and governance standards across

Dev, QA, and Production

, including Git , Branches etc.

- Design data integration patterns for

SAP/ERP, databases, APIs, files, streaming platforms, and cloud services

.

- Design and evaluate data models and serving patterns for

BI, analytics, Data Science, AI, applications, and downstream consumers

.

- Establish standards for Databricks compute, performance optimization, monitoring, reliability, and cost management

.

- Proactively identify and resolve architecture, security, governance,



performance, and production issues.

- Work closely with

Security, Cloud, Infrastructure, Data Governance, Data Science, BI, application, and business teams to deliver enterprise solutions.

- Provide technical leadership, architecture reviews, hands-on prototyping, engineering standards, and mentoring.

- Demonstrate a strong ownership mindset by independently driving initiatives, managing dependencies and risks, and ensuring successful delivery.

You Must Have

- 9+ years of Data Engineering/Data Architecture experience with

4+ years of hands-on Databricks experience

.

- Strong experience with

Databricks, Delta Lake, Spark, SQL, Python/PySpark, and Lakehouse architecture

.

- Strong knowledge of

Unity Catalog, Data Governance, Data Security, RBAC, lineage, auditing, data quality, and access controls

.

- Experience designing enterprise batch, streaming, and reusable data pipelines

.

- 1+ years of Experience with

Databricks AI/BI, Genie, Apps, and other emerging services

.

- Strong AWS experience, including

S3, Lambda, Glue, CloudWatch, RDS, and Redshift

.

- Solid experience in

Git, Dev/QA/Production deployment patterns

- Strong understanding of data modeling, performance optimization, scalability, reliability, security, and cost management

.

- Proven ability to own complex technical initiatives

, solve production issues, and work effectively across multiple technical and business teams.

- Excellent communication, architecture, problem-solving, and technical leadership skills.

- Bachelor’s degree in Computer Science, Information Technology, Data Engineering, or a related field.

Nice to Have

- Databricks and/or AWS certifications.

- Experience with dbt, Sigma, Tableau, or Power BI

.

- Experience with enterprise

FinOps, cost/usage monitoring, and platform governance

.

- SAP integration or functional experience.

📌 Databricks Data Engineering Architect (Bengaluru)
🏢 Blue Cloud Softech Solutions
📍 Bengaluru