Airflow SME

Experience: 8+ Years

Role: Airflow SME / Architect

Role Overview

We are looking for an experienced Airflow SME with strong expertise in Apache Airflow architecture, enterprise workflow orchestration, and migration from legacy scheduling platforms such as UC4/Automic.

The candidate should have hands-on experience designing and implementing Airflow in enterprise environments, along with strong experience in job scheduling, orchestration, migration, Kubernetes/OpenShift, and modernization programs.

Key Responsibilities

- Provide expert guidance on Apache Airflow capabilities, architecture, implementation, and limitations.
- Define the Airflow implementation roadmap, adoption strategy, and enterprise standards.
- Assess existing scheduling platforms such as UC4/Automic, Control-M, TWS, Cron, and recommend migration strategies.
- Define the enterprise Airflow reference architecture.
- Design multi-workplace setups across Dev, SIT, UAT, and Production.
- Define High Availability (HA), Disaster Recovery (DR), scalability, and capacity planning strategies.
- Recommend appropriate Airflow executor strategies such as Kubernetes, Celery, or Local Executor.
- Guide Airflow installation, configuration, and production deployment.
- Review Helm-based deployments and validate Kubernetes/OpenShift implementation designs.
- Define and review Airflow metadata database architecture and infrastructure readiness.
- Establish DAG development standards, naming conventions, coding standards, and reusable components.




- Review DAG design and code quality to ensure scalability, maintainability, reliability, and performance.
- Assess existing UC4/Automic jobs and define migration methodology and conversion approaches.
- Validate migrated DAGs and support testing, cutover planning, and migration issue remediation.
- Define RBAC, access management, secrets management, and authentication requirements.
- Review Entra ID/LDAP integration approaches.
- Support audit, compliance, security, and governance requirements.
- Review test plans, test scenarios, and overall migration readiness.

Mandatory Skills
- Strong experience in Apache Airflow Architecture and Implementation.
- Strong experience in Enterprise Workflow Orchestration and Job Scheduling.
- Strong experience with UC4/Automic and migration/modernization.
- Experience with Airflow implementation on Kubernetes/OpenShift.
- Experience in enterprise-scale migration programs.
- Strong understanding of Airflow DAGs, Executors, HA, DR, scalability, and performance.
- Experience with Airflow deployment, configuration, and production support.
- Strong understanding of RBAC, authentication, secrets management, and enterprise security.

Good to Have
- Experience with Control-M, TWS, Cron, or other enterprise schedulers.
- Experience with Helm and Kubernetes/OpenShift deployments.
- Knowledge of Entra ID / LDAP integration.
- Experience with enterprise cloud, DevOps, CI/CD, monitoring, and governance.
- Experience defining Airflow reference architectures and enterprise adoption strategies.

📌 Airflow SME (Bengaluru)
🏢 Spectrum Talent Management
📍 Bengaluru