Description

We are looking for a Senior Databricks Engineer to design, develop, and optimize scalable data solutions using Databricks for large enterprise data platforms. The role requires strong hands on expertise in Spark, PySpark, SQL, and cloud ecosystems, with ownership of end to end data pipelines and mentoring of junior team members in an offshore delivery model.

Key Responsibilities

Data Engineering & Development

- Design and develop scalable data pipelines using Azure Databricks / Databricks on GCP
- Build and optimize ETL/ELT pipelines using Spark, PySpark, and SQL
- Implement Delta Lake for ACID transactions, data versioning, and performance optimization
- Handle structured and semi structured data (JSON, Parquet, Avro)

Cloud & Integration

- Integrate Databricks with Cloud Storage, ADLS / GCS, BigQuery, Snowflake
- Implement secure data access using IAM, service principals, secrets
- Work with CI/CD pipelines for Databricks deployments
- Collaborate with Data Scientists, BI teams, and downstream consumers

Delivery & Offshore Ownership

- Own offshore delivery for Databricks workstreams
- Provide technical guidance and code reviews to junior engineers




- Work closely with onshore counterparts and clients across time zones
- Participate in sprint planning, backlog grooming, and technical estimations

Required Skills

Technical Skills

- 6–10 years of experience in Data Engineering
- 3+ years of robust hands on experience with Databricks
- Strong expertise in Apache Spark, PySpark, Spark SQL
- Experience with Delta Lake
- Proficiency in SQL (query optimization, complex joins)
- Experience with Azure or GCP cloud platforms
- Experience with Git, CI/CD, and DevOps practices

Preferred Skills

- Experience with Azure Data Lake / BigQuery / Snowflake
- Exposure to streaming frameworks (Kafka, Spark Structured Streaming)
- Knowledge of data governance, data quality, and security
- Experience supporting production workloads

Soft Skills

- Strong analytical and problem solving skills
- Ability to own modules independently in an offshore model
- Good communication skills for client and onshore coordination
- Mentoring mindset and ability to guide junior resources

Mandatory skills* Databricks, Python

📌 Databricks, Python (India)
🏢 Clifyx
📍 India