We are looking for an Airflow Architect to own the design, build and long-term evolution of our enterprise orchestration platform, deployed from the ground up across on-premise Data Centre and DR sites. The platform's primary purpose is infrastructure and operations automation patching, provisioning, backups, certificate and credential rotation, compliance checks, health remediation and cross-system runbook execution with data pipeline workloads as a secondary tenant.

The person in this role is the technical authority for Airflow in the organization: they set the architecture, run it like a product with OEM-grade support, size and secure it, and lead the engineering team that operates it. MySQL is our standard RDBMS, so proven ability to design, tune and run Airflow's metadata layer on MySQL is essential.

Role & Responsibilities

# 1. Architecture & Build (DC + DR)

- Define the target-state architecture for Apache Airflow on on-premise infrastructure VMs, bare metal, with no dependency on any managed cloud service.
- Design a highly available topology: multiple active schedulers, redundant webservers/API servers behind a load balancer, segregated Celery executor worker pools, and resilient broker (RabbitMQ) and metadata layers.
- Design and own the DR site: replication approach for the MySQL metadata store, DAG/automation code, connections, variables and logs; define and defend RTO/RPO; author failover, failback and switchover runbooks and drive periodic DR drills with formal sign-off.

# 2. Infrastructure Automation (Primary Workload Focus)

- Architect Airflow as the automation and scheduling fabric for infrastructure operations, replacing scattered cron jobs, shell scripts and legacy schedulers.
- Design reusable, idempotent automation patterns for use cases such as:
- OS and middleware patching campaigns with pre-checks, staged rollout and rollback

# 3.



MySQL Metadata Layer Design & Ownership

- Design the Airflow metadata database on MySQL 8.0
- Ensure the configuration correctly supports multiple active schedulers, which depend on MySQL 8.0 row-locking semantics - # 4. OEM-Style Support for an Open-Source Stack
- Track upstream releases, provider-package changes and security advisories; plan and execute version upgrades with minimal or zero downtime.
- Maintain internal builds constraint/requirements pinning, and any internally maintained patches or forks; contribute fixes upstream where appropriate.
- Own the knowledge base: architecture documents, HLD/LLD, SOPs, runbooks, onboarding guides and troubleshooting playbooks.
- Provide design consultancy and review to teams onboarding automation onto the platform; enforce standards and reject anti-patterns.

# 5. Capacity Planning & Performance

- Forecast capacity from DAG count, task volume, concurrency profiles, target-fleet size and growth projections; size worker pools, CPU/memory, MySQL connections and broker throughput accordingly.

# 6. Security & Vulnerability Management

- Own the vulnerability lifecycle for the full stack Airflow core and providers, Python dependencies, base OS and container images, MySQL, RabbitMQ and supporting services.
- Monitor NVD/CVE feeds, CISA KEV, Apache and vendor advisories; assess exploitability and applicability, and remediate within organisational patch SLAs.

# 7. Leadership

- Lead a team of 8 platform and support engineers




- Represent the platform in architecture review boards, CAB/change management and senior stakeholder forums; present designs, risks, trade-offs and roadmap.
- Drive automation-first culture and continuous reduction of manual toil year over year.

Preferred candidate profile

Qualification:

- Bachelor's or Master's in Computer Science, Engineering or an equivalent discipline.
- 8–10 years in infrastructure engineering, platform engineering, DevOps or SRE, with demonstrable Apache Airflow architecture and operations experience (Airflow 2.x; 3.x exposure a plus).
- Proven record of building Airflow from scratch on-premise, including HA and a tested DR setup — not solely managed-service usage.
- Solid MySQL 8.x design and administration: replication, InnoDB Cluster/Group Replication, GTID, binlog and PITR, connection pooling, performance tuning.
- Strong Python; able to read Airflow source and build custom operators, hooks, sensors and plugins.
- Deep Linux administration and shell scripting; Windows automation exposure a plus.
- Hands-on Ansible (and/or AWX/Tower) for infrastructure automation, plus Terraform and Git-based workflows.

Good to Have

- Migration experience from legacy schedulers — Control-M, Autosys, Tidal, Rundeck, cron — onto Airflow.
- Contributions to Apache Airflow or related open-source projects.
- ServiceNow / ITSM and CMDB integration experience.
- Storage, backup (Veeam, NetBackup, Commvault), virtualisation (VMware) or network automation exposure.
- Vault, Keycloak, LDAP/Kerberos integration.
- Certifications: Astronomer Certified Airflow, CKA/CKAD, RHCE, Red Hat Ansible Automation, MySQL DBA, or a security certification.

Regulated-industry background (BFSI, telecom, healthcare) with formal DR and audit obligations. Interested candidates please share your resume to [email protected]

📌 Airflow Architect (Chennai)
🏢 Trizenai Technologies
📍 Chennai