About the Role

Lead the architecture, design and delivery of enterprise-scale Databricks data platforms across Azure, AWS or GCP. The role will provide technical leadership for modern data engineering, Lakehouse architecture, data integration, data migration, analytics and AI/ML workloads while ensuring scalable, secure, performant and cost-effective solutions.

Responsibilities

- Design and implement enterprise-scale Databricks Lakehouse architectures.
- Define architecture across Azure, AWS and GCP based on business and technical requirements.
- Design data ingestion, transformation and processing using Databricks, Spark, PySpark, SQL and Python.
- Establish Lakehouse architecture, including Delta Lake, Medallion Architecture and data modelling.
- Design batch and real-time data processing and integration solutions.
- Architect and implement Databricks Workflows, Jobs, Unity Catalog and governance capabilities.
- Define data security, access control, lineage, quality, monitoring and governance standards.
- Design scalable and optimized solutions for performance and cloud cost.
- Lead migration and modernization of legacy data platforms to Databricks.
- Integrate Databricks with cloud-native services across Azure / AWS / GCP.
- Establish CI/CD, Git, DevOps and deployment standards for data engineering.
- Provide technical leadership through design reviews, code reviews and engineering standards.




- Guide and mentor Data Engineers and technical teams.
- Conduct POCs and evaluate new Databricks/cloud capabilities.
- Work with business stakeholders, project managers and enterprise architects to translate requirements into technical solutions.
- Support estimation, solution proposals, technical presentations and client discussions.
- Adhere to the Information Security Management policies and procedures.

Required Skills

- 10+ years of experience in Data Engineering / Data Architecture.
- Strong hands-on experience with Databricks and Lakehouse architecture.
- Strong knowledge of Apache Spark, PySpark, Python and SQL.
- Experience with Delta Lake and Medallion Architecture.
- Strong experience in data modelling, ETL/ELT and data integration.
- Experience designing scalable batch and real-time data solutions.
- Robust knowledge of at least one major cloud platform: Azure, AWS or GCP.
- Understanding of cloud data services and integration patterns.
- Experience with Databricks Workflows/Jobs and Unity Catalog.
- Strong understanding of data security, governance, quality and lineage.
- Experience with CI/CD, Git and DevOps practices.
- Strong solution architecture and technical leadership skills.

Preferred Skills /Positive to Have

- Snowflake
- Azure Data Factory / AWS Glue / GCP Dataflow
- Kafka / Event Hubs / Kinesis / Pub/Sub
- Databricks ML / MLflow

📌 Data Architect (Kochi)
🏢 Experion Technologies
📍 Kochi