About the Role:

We are seeking a Senior Databricks Data Engineer to design and develop scalable data pipelines, optimize Spark workloads, and build enterprise-grade data engineering solutions using Azure Databricks. The engineer will play a critical role in migrating Finance data from on-premises SAP systems into a up-to-date cloud Lakehouse architecture.

Key Responsibilities

- Design and develop scalable ETL/ELT pipelines.

- Develop PySpark and Spark SQL transformations.

- Build Bronze, Silver, and Gold layer data pipelines.

- Integrate SAP and enterprise source systems.

- Implement Delta Lake best practices.

- Optimize Spark jobs for performance and cost.

- Build reusable data engineering frameworks.

- Implement data quality and validation rules.

- Develop incremental and CDC pipelines.

- Collaborate with BI developers for optimized reporting datasets.

- Participate in code reviews and technical documentation.

- Support production deployments and troubleshooting.

📌 Data Engineer with Data Bricks (Mumbai)
🏢 Exponentia.ai
📍 Mumbai