Senior Data Engineer – Databricks / PySpark / ADF / Logic Apps / Airflow

Experience: 7+ Years

Employment Type: C2C

Work Mode: Hybrid – Pan India

Shift: 11:00 AM – 9:00 PM IST

Job Overview

We are looking for a Senior Data Engineer with strong hands-on experience in Databricks, PySpark, Azure Data Factory (ADF), Azure Logic Apps, and Apache Airflow.

The candidate will be responsible for designing and developing scalable ETL/ELT pipelines, workflow orchestration, Azure data solutions, and Lakehouse platforms.

Mandatory Skills

- 7+ years of Data Engineering experience
- Databricks & PySpark
- Azure Data Factory (ADF) – Strong hands-on
- Azure Logic Apps – Solid hands-on
- Apache Airflow
- Delta Lake / Lakehouse Architecture
- Strong Python & SQL
- ETL/ELT pipeline development
- Azure Cloud services
- Workflow orchestration and dependency management
- Pipeline monitoring, logging, alerting and troubleshooting

Key Responsibilities

- Design and develop scalable data pipelines using Databricks, PySpark, ADF, Logic Apps and Airflow
- Build and manage Airflow DAGs, ADF pipelines and Logic Apps workflows
- Implement scheduled,



event-driven and micro-batch workflows
- Develop high-performance ETL/ELT pipelines using Delta Lake and Lakehouse architecture
- Integrate with ADLS Gen2, Blob Storage, APIs and Azure services
- Implement monitoring, retries, alerts and failure-handling mechanisms
- Optimize pipelines for performance, scalability and cost
- Implement CI/CD using GitHub Actions / Azure DevOps
- Support data quality, governance and security requirements
- Enable AI/ML and GenAI initiatives through reliable, ML-ready datasets
- Collaborate with Data Science, Analytics and Business teams

Preferred Skills

- AI/ML, LLM, GenAI or MLflow pipeline exposure
- Event-driven architecture and API integrations
- Data Governance
- Azure / Databricks certifications

Must Have Databricks + PySpark | ADF | Azure Logic Apps | Airflow | Delta Lake | Python | Advanced SQL | ETL/ELT | Azure Cloud

Immediate / Short Notice candidates preferred.

Pay: ₹90,000.00 - ₹110,000.00 per month

Work Location: Hybrid remote in Hyderabad, Telangana

📌 Senior Data Engineer – Databricks / PySpark / ADF / Logic Apps / Airflow (Hyderabad)
🏢 Technology next
📍 Hyderabad