Key Responsibilities
Design and develop scalable data pipelines using Databricks and PySpark.
Build and optimize ETL/ELT workflows for large-volume data processing.
Develop data transformations using PySpark and SQL.
Work extensively with Delta Lake for reliable and scalable data storage.
Develop and maintain data pipelines using Databricks Workflows/Lakeflow.
Perform data integration from multiple structured and unstructured sources.
Optimize Spark jobs, queries, and pipeline performance.
Implement data quality, validation, monitoring, and error-handling mechanisms.
Collaborate with Data Architects, BI teams, and business stakeholders.
Follow best practices for CI/CD, version control, and deployment.

Required Skills
Strong hands-on experience with Databricks




Robust PySpark and SQL skills
Valuable understanding of Delta Lake
Strong experience in Data Engineering and ETL
Understanding of Data Warehousing and Data Modelling
Experience with cloud platforms such as Azure / AWS / GCP
Knowledge of Git, CI/CD and DevOps
Good analytical and problem-solving skills
Robust communication and stakeholder management skills

Good to Have
Experience with Unity Catalog
Databricks certifications
Experience with Kafka / Azure Data Factory / Event Hubs
Knowledge of Lakehouse architecture
Experience in migrating legacy data platforms to Databricks

📌 Data Engineer And Senior Data Engineer Mumbai
🏢 Celebal Technologies
📍 Mumbai