Experience: 8+ Years

Role: Senior Databricks Developer

Job Summary

We are looking for an experienced Senior Databricks Developer with 8+ years of overall experience and strong hands-on expertise in designing, developing, and optimizing enterprise-scale data engineering solutions using Databricks.

The ideal candidate should have strong proficiency in Databricks, PySpark, Python, SQL, Delta Lake, Unity Catalog, data modelling, and ETL/ELT pipelines. Experience in modernizing legacy data warehouses and migrating traditional ETL workloads to Databricks is essential.

Key Responsibilities

- Design, develop, and optimize scalable data pipelines using Databricks, PySpark, Python, and SQL.
- Implement Medallion Architecture (Bronze, Silver, and Gold layers).
- Develop reusable frameworks for data ingestion, transformation, validation, and processing.
- Design, implement, and optimize Delta Lake tables and large-scale Spark workloads.
- Build and manage data workflows using Databricks Workflows.
- Implement data governance and security using Unity Catalog, including RBAC, row/column-level security, and data masking.
- Develop dimensional, reference, master, and curated data models.
- Modernize legacy data warehouse solutions from SQL Server, Oracle, and Teradata to Databricks.




- Migrate stored procedures and traditional ETL processes into scalable cloud-based ETL/ELT pipelines.
- Perform Spark and SQL performance tuning, cost optimization, troubleshooting, and root-cause analysis.
- Implement pipeline monitoring, error handling, and recovery mechanisms.
- Support CI/CD, code reviews, testing, deployment, and production support.
- Mentor junior developers and participate in technical and design discussions.
- Collaborate with architects, business stakeholders, and onshore/offshore delivery teams.

Mandatory Technical Skills
- Databricks
- PySpark
- Python
- Advanced SQL
- Delta Lake
- Unity Catalog
- Medallion Architecture (Bronze/Silver/Gold)
- ETL/ELT Development
- Data Modelling
- Spark Performance Tuning
- Databricks Workflows

Good to Have
- Databricks AI capabilities.
- Experience with SQL Server, Oracle, and Teradata migration.
- Cloud data engineering experience.
- CI/CD and production support.

Desired Candidate Profile
- 8+ years of overall IT experience.
- Strong hands-on experience in Databricks development.
- Experience in enterprise-scale data engineering and warehouse modernization.
- Robust problem-solving, analytical, and communication skills.
- Ability to work independently and mentor junior team members.

📌 Senior Databricks Developer II Immediate II Remote (Noida)
🏢 Compunnel
📍 Noida