Job Summary

We are looking for a skilled Databricks Developer with experience in data engineering, data transformation, and building scalable data pipelines using Azure Databricks and Apache Spark.

Key Responsibilities
Design, develop, and maintain data pipelines using Azure Databricks and PySpark.
Develop ETL/ELT processes for data ingestion, transformation, and integration.
Work with Python, SQL, Apache Spark, and Delta Lake.
Implement data lakehouse solutions and optimize data processing performance.
Integrate data from multiple sources and ensure data quality.
Collaborate with data engineers, architects, and business teams to deliver data solutions.
Troubleshoot pipeline failures and optimize jobs for performance and cost.
Implement data security, governance, and access controls.

Required Skills




Hands-on experience with Azure Databricks and PySpark.
Solid knowledge of Python and SQL.
Experience with Apache Spark, Delta Lake, and ETL/ELT pipelines.
Understanding data warehousing and lakehouse architecture.
Experience with Azure Data Factory, ADLS, or other cloud data services.
Knowledge of Git, CI/CD, and workflow orchestration.
Solid analytical and problem-solving skills.

Preferred Qualifications
Experience with Databricks Unity Catalog and Delta Live Tables (Lakeflow Declarative Pipelines).
Knowledge of Azure cloud services and data governance.
Experience in performance tuning and production deployment of data pipelines.

📌 Lead Data Engineer Mumbai (India)
🏢 IRT Digital Analytics Solutions
📍 India