Experience: 8+ Years

Role: Senior Databricks Developer

Job Summary

We are looking for an experienced Senior Databricks Developer with 8+ years of overall experience and strong hands-on expertise in designing, developing, and optimizing enterprise-scale data engineering solutions using Databricks.

The ideal candidate should have robust proficiency in Databricks, PySpark, Python, SQL, Delta Lake, Unity Catalog, data modelling, and ETL/ELT pipelines. Experience in modernizing legacy data warehouses and migrating traditional ETL workloads to Databricks is essential.

Key Responsibilities
Design, develop, and optimize scalable data pipelines using Databricks, PySpark, Python, and SQL.
Implement Medallion Architecture (Bronze, Silver, and Gold layers).
Develop reusable frameworks for data ingestion, transformation, validation, and processing.
Design, implement, and optimize Delta Lake tables and large-scale Spark workloads.
Build and manage data workflows using Databricks Workflows.
Implement data governance and security using Unity Catalog, including RBAC, row/column-level security, and data masking.
Develop dimensional, reference, master, and curated data models.
Modernize legacy data warehouse solutions from SQL Server, Oracle, and Teradata to Databricks.




Migrate stored procedures and traditional ETL processes into scalable cloud-based ETL/ELT pipelines.
Perform Spark and SQL performance tuning, cost optimization, troubleshooting, and root-cause analysis.
Implement pipeline monitoring, error handling, and recovery mechanisms.
Support CI/CD, code reviews, testing, deployment, and production support.
Mentor junior developers and participate in technical and design discussions.
Collaborate with architects, business stakeholders, and onshore/offshore delivery teams.

Mandatory Technical Skills
Databricks
PySpark
Python
Advanced SQL
Delta Lake
Unity Catalog
Medallion Architecture (Bronze/Silver/Gold)
ETL/ELT Development
Data Modelling
Spark Performance Tuning
Databricks Workflows

Positive to Have
Databricks AI capabilities.
Experience with SQL Server, Oracle, and Teradata migration.
Cloud data engineering experience.
CI/CD and production support.

Desired Candidate Profile
8+ years of overall IT experience.
Strong hands-on experience in Databricks development.
Experience in enterprise-scale data engineering and warehouse modernization.
Robust problem-solving, analytical, and communication skills.
Ability to work independently and mentor junior team members.

📌 Senior Databricks Developer Ii Immediate Ii Remote Noida (India)
🏢 Compunnel
📍 India