Key Responsibilities

- Design and develop scalable data pipelines using Databricks and PySpark.

- Build and optimize ETL/ELT workflows for large-volume data processing.

- Develop data transformations using PySpark and SQL.

- Work extensively with Delta Lake for reliable and scalable data storage.

- Develop and maintain data pipelines using Databricks Workflows/Lakeflow.

- Perform data integration from multiple structured and unstructured sources.

- Optimize Spark jobs, queries, and pipeline performance.

- Implement data quality, validation, monitoring, and error-handling mechanisms.

- Collaborate with Data Architects, BI teams, and business stakeholders.

- Follow best practices for CI/CD, version control, and deployment.

Required Skills

- Strong hands-on experience with Databricks

- Solid PySpark and SQL skills





- Good understanding of Delta Lake

- Strong experience in Data Engineering and ETL

- Understanding of Data Warehousing and Data Modelling

- Experience with cloud platforms such as Azure / AWS / GCP

- Knowledge of Git, CI/CD and DevOps

- Good analytical and problem-solving skills

- Strong communication and stakeholder management skills

Good to Have

- Experience with Unity Catalog

- Databricks certifications

- Experience with Kafka / Azure Data Factory / Event Hubs

- Knowledge of Lakehouse architecture

- Experience in migrating legacy data platforms to Databricks

📌 Data Engineer- Gurugram Immediate joiner only
🏢 Celebal Technologies
📍 Gurugram