Roles and Responsibilities:

- Design, develop, and optimize data pipelines using Apache Spark on Databricks.
- Implement ETL/ELT workflows to ingest, transform, and load data from various sources.
- Collaborate with data scientists, analysts, and business stakeholders to deliver high-quality data solutions.
- Ensure data quality, integrity, and governance across all pipelines.
- Monitor and troubleshoot performance issues in Databricks notebooks and jobs.
- Integrate Databricks with cloud services (Azure Data Lake) and enterprise data platforms.
- Participate in code reviews, documentation, and agile ceremonies.

Qualification and Experience:

- 5+ years of experience in data engineering or big data development.
- Strong proficiency in Apache Spark, PySpark, and SQL.
- Hands-on experience with Databricks platform (including notebooks, jobs, clusters).
- Experience with cloud platforms (Azure) and cloud-native data services.
- Familiarity with Delta Lake, MLflow, and Databricks Workflows.
- Knowledge of CI/CD practices and version control (Git).
- Solid problem-solving and communication skills.

📌 Databricks engineer (Gurugram)
🏢 XCEEDANCE
📍 Gurugram