Senior Data Engineer – Databricks / PySpark / ADF / Logic Apps / Airflow

Experience: 7+ Years
Employment Type: C2C
Work Mode: Hybrid – Pan India
Shift: 11:00 AM – 9:00 PM IST

Job Overview

We are looking for a Senior Data Engineer with solid hands-on experience in Databricks, PySpark, Azure Data Factory (ADF), Azure Logic Apps, and Apache Airflow.

The candidate will be responsible for designing and developing scalable ETL/ELT pipelines, workflow orchestration, Azure data solutions, and Lakehouse platforms.

Mandatory Skills
7+ years of Data Engineering experience
Databricks & PySpark
Azure Data Factory (ADF) – Strong hands-on
Azure Logic Apps – Solid hands-on
Apache Airflow
Delta Lake / Lakehouse Architecture
Solid Python & SQL
ETL/ELT pipeline development
Azure Cloud services
Workflow orchestration and dependency management
Pipeline monitoring, logging, alerting and troubleshooting

Key Responsibilities
Design and develop scalable data pipelines using Databricks, PySpark, ADF, Logic Apps and Airflow
Build and manage Airflow DAGs, ADF pipelines and Logic Apps workflows
Implement scheduled,



event-driven and micro-batch workflows
Develop high-performance ETL/ELT pipelines using Delta Lake and Lakehouse architecture
Integrate with ADLS Gen2, Blob Storage, APIs and Azure services
Implement monitoring, retries, alerts and failure-handling mechanisms
Optimize pipelines for performance, scalability and cost
Implement CI/CD using GitHub Actions / Azure DevOps
Support data quality, governance and security requirements
Enable AI/ML and GenAI initiatives through reliable, ML-ready datasets
Collaborate with Data Science, Analytics and Business teams

Preferred Skills
AI/ML, LLM, GenAI or MLflow pipeline exposure
Event-driven architecture and API integrations
Data Governance
Azure / Databricks certifications

Must Have

Databricks + PySpark | ADF | Azure Logic Apps | Airflow | Delta Lake | Python | Advanced SQL | ETL/ELT | Azure Cloud

Immediate / Short Notice candidates preferred.

Pay: ₹90,000.00 - ₹110,000.00 per month

Work Location: Hybrid remote in Hyderabad, Telangana

📌 Senior Data Engineer – Databricks / Pyspark / Adf / Logic Apps / Airflow Hyderabad
🏢 Technology next
📍 Hyderabad