Job Title: Airflow And Databricks SME

nJob Location: Hyderabad

nExperiences: 5+ Years

nJob Type: Full-time

nShift Timings: Rotational Shifts (24/7)

nWork Mode: 5 Days Work from Office

n

nJob Summary:

nWe are looking for a 5 to 8 years of experienced Airflow & Databricks Scheduling SME to manage, support, optimize, and govern enterprise scheduling platforms and workflow orchestration processes. The candidate will be responsible for designing and maintaining Airflow workflows, scheduling Databricks jobs, monitoring critical business data pipelines, resolving production issues, and ensuring SLA compliance across the enterprise data ecosystem.The role requires robust expertise in Apache Airflow, Azure Databricks, workflow scheduling, incident management, problem management, and production support operations.

n

nScheduling & Workflow Management

nDesign, develop, and manage enterprise scheduling workflows using ApacheAirflow and Databricks

nCreate, monitor, and maintain Airflow DAGs for batch and real-time data processing.

nSchedule and orchestrate Databricks notebooks, jobs, and workflows.

nManage complex job dependencies and event-based scheduling.

nEnsure successful execution of daily, weekly, and monthly business-critical jobs.

nMonitor job completion, SLA adherence, and schedule compliance.

nValidate upstream and downstream job dependencies.

nConfigure and manage Databricks Job Workflows.

nMonitor cluster performance and job execution.





nTroubleshoot failed Databricks jobs and Spark execution issues.

nCoordinate with Data Engineering teams for workflow enhancements.

nOptimize Databricks job execution time and resource consumption.

nConfigure and manage Databricks Job Workflows.

nSchedule Databricks notebooks and ETL pipelines using Airflow.

nMonitor cluster performance and job execution.

nTroubleshoot failed Databricks jobs and Spark execution issues.

nCoordinate with Data Engineering teams for workflow enhancements.

nOptimize Databricks job execution time and resource consumption.

n

nKey Responsibilities Summary

nThe Scheduling SME will:

nOwn Airflow and Databricks scheduling platforms.

nEnsure successful execution of enterprise data workflows.

nMonitor and support business-critical batch processing.

nHandle incidents, outages, and escalations.

nLead Major Incident Management activities related to scheduling failures.

nPerform RCA and Problem Management.

nDrive automation and platform optimization.

nEnsure SLA adherence and operational excellence.

nCollaborate with Data Engineering, Cloud, Infrastructure, and Operations teams

n

nMandatory Skills

nApache Airflow, Azure Databricks

nWorkflow Scheduling & Orchestration

nPython, PySpark, SQL

nAzure Cloud, Linux/Unix, Git

nMonitoring Tools (Splunk, Grafana, Dynatrace)

n

nEducation & Certifications:

nBachelor’s degree in Computer Science or related field

📌 Airflow And Databricks SME (Hyderabad)
🏢 Solugenix
📍 Hyderabad