Databricks -Data Engineers :

Job Description: Data Engineer – Databricks (6–8 Years)

Role Summary

We are looking for a Data Engineer with strong Databricks expertise to design and build scalable data pipelines on a contemporary Lakehouse architecture. The ideal candidate should have hands-on experience with Apache Spark, Delta Lake, and cloud data platforms, and be capable of delivering robust, production-grade data solutions.

Role focus: Pipeline engineering + Lakehouse architecture + performance optimisation

Key Responsibilities

Design, build, and optimize scalable data pipelines using Databricks (batch & streaming)

Develop data processing frameworks using PySpark / Spark SQL

Implement Delta Lake-based data storage ensuring ACID compliance and reliability

Develop and maintain medallion architecture (Bronze, Silver, Gold) data layers

Perform data ingestion from multiple sources (APIs, Kafka, cloud storage, databases)

Build data models for analytics and reporting use cases

Ensure data quality, validation, and reconciliation mechanisms

Optimise performance using partitioning,



caching, and query tuning

Implement data governance using Unity Catalog / equivalent tools

Integrate pipelines with orchestration tools (Airflow / ADF / Composer)

Collaborate with stakeholders for requirements, design, and delivery

Enterprise alignment:

Databricks roles focus on “designing, building, and managing scalable data pipelines using Spark and Delta Lake” [1]1

Mandatory Skills (Core Must-Have)

Databricks & Big Data

Strong hands-on Databricks (Azure / GCP)-Primarily GCP preferred

Expertise in Apache Spark (PySpark)

Experience with Delta Lake (ACID, schema evolution)

Programming & Querying

Strong Python + SQL

Data Engineering Fundamentals

ETL/ELT pipeline development

Data modelling (dimensional / Lakehouse)

Batch + streaming data processing

Cloud Platform (One of)

GCP (BigQuery, Dataflow, Composer)

Good to Have Skills

CI/CD (Git, Azure DevOps, GitHub Actions)

I

📌 Databricks- DataBase Migration (Hyderabad)
🏢 Shyena Tech Yarns
📍 Hyderabad