Description :

Industry & Sector : Cloud Data & Analytics consulting focused on Microsoft Azure delivering enterprise data platforms, analytics, and AI-ready data lakes for customers across financial services, retail, and technology.
We build production-grade, secure, and highly performant data pipelines and analytics solutions.

Primary Title : Azure Databricks Engineer.

Location : India In office (client-delivery focused).

Role & Responsibilities :
Design, build and maintain scalable ETL/ELT data pipelines on Azure Databricks using PySpark and Apache Spark for batch and streaming workloads.
Implement and optimize Delta Lake architectures (schema evolution, partitioning, compaction, time travel) to ensure reliable data lakes and high query performance.
Integrate Databricks jobs with Azure Data Factory for orchestration; implement parameterized pipelines, job scheduling, and failure/retry strategies.
Ingest and transform data from ADLS Gen2, Event Hubs/Streaming sources, and relational systems; define data models and effective Parquet/Delta storage strategies.
Establish CI/CD practices for notebooks and infrastructure using Git and Azure DevOps; automate deployments, unit tests, and environment promotion.
Monitor and tune Spark job performance, troubleshoot production issues,



and build observability (logging, metrics, alerts) and runbooks to meet SLAs.

Skills & Qualifications :

Must-Have :
5+ years of hands-on experience delivering data engineering solutions on Azure with robust expertise in Azure Databricks and Apache Spark (PySpark).
Proven experience with Delta Lake, data modeling, partitioning strategies, and Spark performance tuning.
Practical knowledge integrating Databricks with Azure Data Factory and ADLS Gen2 for end-to-end ETL/ELT workflows.
Proficiency in SQL and Python notebook development; experience with Git-based source control and Azure DevOps CI/CD pipelines.
Experience operating production data platforms : monitoring, alerting, job orchestration, and creating operational runbooks.
Ability to work onsite in India and collaborate directly with cross-functional engineering and client teams.

Preferred :
Experience with Scala and building streaming data pipelines.
Familiarity with Apache Airflow, Azure Synapse Analytics, or Terraform for orchestration and infra-as-code.
Databricks or Azure data engineering certifications (Databricks Certified / Azure Data Engineer Associate) a plus.

📌 Azure Databricks Engineer Tamil Nadu
🏢 Zorba consulting
📍 Tamil Nadu