01 Sep
|
Blue Cloud Softech Solutions
|
India
01 Sep
Blue Cloud Softech Solutions
India
Databricks Data Engineering Architect
Primary Responsibilities
- Define and drive the Databricks Data Engineering architecture and strategy , aligned with enterprise Data & Analytics standards.
- Take end-to-end ownership of Databricks solutions from architecture and design through implementation, production deployment, support, and optimization.
- Define and enforce policy-based governance and cost controls across Databricks workloads, including Apps, Agents, compute, AI/ML resources, and AI Gateway .
- Establish and enforce RBAC and least-privilege security across Unity Catalog, Databricks Apps, Agents, AI Gateway, service principals, and Dev/QA/Production environments.
- Design reusable data ingestion and transformation frameworks using Databricks, Spark, Delta Lake, SQL, Python/PySpark, and Lakeflow .
- Architect scalable Lakehouse solutions on Databricks and AWS supporting batch, streaming, near-real-time, analytics, and AI use cases.
- Design and support Agentic AI and Databricks Apps development , including architecture, deployment, integration, security, monitoring,
and production readiness.
- Establish and enforce Data Engineering Governance and Security using Unity Catalog, including RBAC, least privilege, service principals, data access controls, lineage, auditing, and data quality.
- Define engineering and governance standards across Dev, QA, and Production , including Git , Branches etc.
- Design data integration patterns for SAP/ERP, databases, APIs, files, streaming platforms, and cloud services .
- Design and evaluate data models and serving patterns for BI, analytics, Data Science, AI, applications, and downstream consumers .
- Establish standards for Databricks compute, performance optimization, monitoring, reliability, and cost management .
- Proactively identify and resolve architecture, security, governance, performance, and production issues.
- Work closely with Security, Cloud, Infrastructure, Data Governance, Data Science, BI, application, an
📌 Databricks Data Engineering Architect (India)
🏢 Blue Cloud Softech Solutions
📍 India