:

Role & Responsibilities :

- Design, develop, and maintain scalable data pipelines for data ingestion, processing and storage.
- Build and optimize data architectures and data models (Lakehouse / medallion, dimensional) for efficient data storage and retrieval.
- Develop ETL/ELT processes to transform and load data from various sources into data warehouses and data lakes.
- Build and orchestrate pipelines on Azure Databricks using PySpark, Spark SQL and Delta Lake, orchestrated with Databricks Workflows.
- Integrate data from enterprise source systems including SAP (ABAP/CDS extracts, RPA/CSV or connectors) and load into Snowflake and Databricks.
- Own end-to-end CI/CD for data pipelines using Databricks Asset Bundles (DAB) and Azure DevOps (Git repositories, YAML build and release pipelines), promoting code across dev, QA and production.
- Implement data quality, validation, freshness and reconciliation checks with pipeline observability.
- Ensure data integrity, quality, and security across all data systems.
- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs.




- Monitor and troubleshoot data pipelines and workflows to ensure high availability and performance.
- Document data processes, architectures, and data flow diagrams.

Preferred candidate profile :

- 7 - 8 years of hands-on data engineering experience building and running production data pipelines at scale.
- Strong expertise in Azure and Azure data services (ADLS Gen2, Azure Databricks, Azure DevOps).
- Deep hands-on experience with Databricks : PySpark, Spark SQL, Delta Lake, Lakehouse / medallion architecture and Databricks Workflows.
- CI/CD for data engineering using Databricks Asset Bundles (DAB) and Azure DevOps (Git, YAML build/release pipelines, multi-environment promotion). (Must-have)
- Solid Snowflake experience (data modeling, performance tuning, loading and optimization).
- Proficiency in SQL and Python.
- Experience integrating data from SAP and other enterprise ERP / source systems into a data lake or warehouse.
- Solid data modeling (dimensional, star/snowflake, Lakehouse) and ETL/ELT design.
- Building data-quality, validation, reconciliation and pipeline monitoring / observability.

📌 Senior Data Engineer - Azure Databricks (India)
🏢 LeewayHertz Technologies
📍 India