Databricks -Data Engineers :

Job Description: Data Engineer – Databricks (6–8 Years)

Role Summary

We are looking for a Data Engineer with strong Databricks expertise to design and build scalable data pipelines on a modern Lakehouse architecture. The ideal candidate should have hands-on experience with Apache Spark, Delta Lake, and cloud data platforms, and be capable of delivering robust, production-grade data solutions.

Role focus: Pipeline engineering + Lakehouse architecture + performance optimisation

Key Responsibilities

Design, build, and optimize scalable data pipelines using Databricks (batch & streaming)

Develop data processing frameworks using PySpark / Spark SQL

Implement Delta Lake-based data storage ensuring ACID compliance and reliability

Develop and maintain medallion architecture (Bronze, Silver, Gold) data layers

Perform data ingestion from multiple sources (APIs, Kafka, cloud storage, databases)

Build data models for analytics and reporting use cases

Ensure data quality, validation, and reconciliation mechanisms

Optimise performance using partitioning, caching, and query tuning

Implement data governance using Unity Catalog / equivalent tools

Integrate pipelines with orchestration tools (Airflow / ADF / Composer)

Collaborate with stakeholders for requirements, design, and delivery

Enterprise alignment:

Databricks roles focus on “designing, building, and managing scalable data pipelines using Spark and Delta Lake” [1]1

Mandatory Skills (Core Must-Have)

Databricks & Big Data





Strong hands-on Databricks (Azure / GCP)-Primarily GCP preferred

Expertise in Apache Spark (PySpark)

Experience with Delta Lake (ACID, schema evolution)

Programming & Querying

Strong Python + SQL

Data Engineering Fundamentals

ETL/ELT pipeline development

Data modelling (dimensional / Lakehouse)

Batch + streaming data processing

Cloud Platform (One of)

GCP (BigQuery, Dataflow, Composer)

Valuable to Have Skills

CI/CD (Git, Azure DevOps, GitHub Actions)

Infrastructure as Code (Terraform)

Containerisation (Docker, Kubernetes)

Data orchestration (Airflow, Composer)

Experience in real-time streaming (Kafka, Event Hub)

Exposure to BI tools (Power BI, Tableau)

Databricks-Specific Expectations

Deep understanding of Lakehouse architecture

Working knowledge of:

Delta tables (MERGE, UPSERT, Time Travel)

Job orchestration (Workflows)

Cluster management & optimisation

Exposure to Unity Catalog (data governance)

Enterprise reference:

Databricks certifications validate ingestion, transformation, governance, and performance expertise [1]1

Experience & Qualification

6–8 years of experience in Data Engineering / Big Data

3+ years hands-on experience with Databricks

Bachelor’s/Master’s in Engineering / Computer Science or equivalent

Key Competencies

Strong problem-solving and analytical skills

Ability to work in high-volume, enterprise-grade data environments

Experience in performance tuning and optimisation

Good understanding of data governance and compliance (BFSI preferred)

Pay: From ₹800,000.00 per year

Work Location: In person

📌 Databricks- DataBase Migration (India)
🏢 Shyena Tech Yarns
📍 India