Senior Data Engineer - Databricks | Hyderabad | Immediate joiners to 30 days only

We are hiring an experienced Senior Data Engineer for an enterprise data engineering programme with a leading global client. You will design, build and run large-scale batch and streaming data pipelines on Azure Databricks. This is a hands-on role for someone who has built production pipelines end to end.

Roles & Responsibilities:

- Design and build scalable ETL/ELT pipelines on Azure Databricks using PySpark and Spark SQL

- Build and orchestrate workflows in Azure Data Factory (pipelines, linked services, triggers, parameterisation, monitoring)

- Develop real-time and near-real-time ingestion using Apache Kafka with Spark Structured Streaming

- Implement Delta Lake and medallion (bronze/silver/gold) architecture

- Write complex, high-performance SQL for transformation, validation and reporting

- Tune Spark jobs and clusters for performance and cost

- Turn stakeholder requirements into technical design

- Own testing, CI/CD deployment, production support and data-quality checks





- Document pipelines and mentor junior engineers

Mandatory Skills (hands-on in ALL):

- Azure Databricks (notebooks, jobs/workflows, Delta Lake, clusters)

- PySpark / Spark SQL with performance tuning

- Azure Data Factory - production ETL pipelines end to end

- Advanced SQL (complex joins, window functions, query tuning)

- Kafka - streaming ingestion (producers/consumers, topics, Spark Structured Streaming)

- ETL design, data modelling and data warehousing

Positive to Have:

- Unity Catalog, Delta Live Tables, Databricks Workflows

- ADLS Gen2, Azure Synapse, Event Hubs

- CI/CD with Azure DevOps or GitHub

- Databricks or DP-203 certification

- Airflow or Snowflake

Qualification: B.E./B.Tech/MCA/M.Sc (Computer Science or equivalent) - mandatory. 7+ years total IT experience.

Notice period: Immediate to 30 days only.

How to apply: Share your updated resume with current CTC, expected CTC, notice period and current location.

📌 Senior Data Engineer - Azure Databricks, PySpark, ADF, Kafka (Hyderabad)
🏢 Bendito Soft Tech
📍 Hyderabad