We are looking for an Airflow Architect to own the design, build and long-term evolution of our enterprise orchestration platform, deployed from the ground up across on-premise Data Centre and DR sites. The platform's primary purpose is infrastructure and operations automation — patching, provisioning, backups, certificate and credential rotation, compliance checks, health remediation and cross-system runbook execution — with data pipeline workloads as a secondary tenant.

The person in this role is the technical authority for Airflow in the organization: they set the architecture, run it like a product with OEM-grade support, size and secure it, and lead the engineering team that operates it. MySQL is our standard RDBMS, so proven ability to design, tune and run Airflow's metadata layer on MySQL is essential.

Key Responsibilities

1. Architecture & Build (DC + DR)

- Define the target-state architecture for Apache Airflow on on-premise infrastructure — VMs, bare metal, — with no dependency on any managed cloud service.
- Design a highly available topology: multiple active schedulers, redundant webservers/API servers behind a load balancer, segregated Celery executor worker pools, and resilient broker (RabbitMQ) and metadata layers.

2. Infrastructure Automation (Primary Workload Focus)

- Architect Airflow as the automation and scheduling fabric for infrastructure operations, replacing scattered cron jobs, shell scripts and legacy schedulers.
- Design reusable, idempotent automation patterns for use cases such as:
- OS and middleware patching campaigns with pre-checks, staged rollout and rollback

3. MySQL Metadata Layer Design & Ownership

- Design the Airflow metadata database on MySQL 8.0+ (InnoDB, `utf8mb4`, correct collation, `explicit_defaults_for_timestamp=1`)



and validate compatibility with the target Airflow version.
- Ensure the configuration correctly supports multiple active schedulers, which depend on MySQL 8.0 row-locking semantics (`SELECT ... FOR UPDATE SKIP LOCKED`) — including awareness of Airflow's known MySQL-specific behaviours and limitations versus other backends.

4. OEM-Style Support for an Open-Source Stack

- Track upstream releases, provider-package changes and security advisories; plan and execute version upgrades with minimal or zero downtime.
- Maintain internal builds — constraint/requirements pinning, and any internally maintained patches or forks; contribute fixes upstream where appropriate.

5. Capacity Planning & Performance

- Forecast capacity from DAG count, task volume, concurrency profiles, target-fleet size and growth projections; size worker pools, CPU/memory, MySQL connections and broker throughput accordingly.

6. Security & Vulnerability Management

- Own the vulnerability lifecycle for the full stack — Airflow core and providers, Python dependencies, base OS and container images, MySQL, RabbitMQ and supporting services.
- Monitor NVD/CVE feeds, CISA KEV, Apache and vendor advisories; assess exploitability and applicability, and remediate within organisational patch SLAs.

Must-Have Skills
- 8–10 years in infrastructure engineering, platform engineering, DevOps or SRE,



with demonstrable Apache Airflow architecture and operations experience (Airflow 2.x; 3.x exposure a plus).
- Proven record of building Airflow from scratch on-premise, including HA and a tested DR setup — not solely managed-service usage.
- Robust MySQL 8.x design and administration: replication, InnoDB Cluster/Group Replication, GTID, binlog and PITR, connection pooling, performance tuning.
- Strong Python; able to read Airflow source and build custom operators, hooks, sensors and plugins.
- Deep Linux administration and shell scripting; Windows automation exposure a plus.
- Hands-on Ansible (and/or AWX/Tower) for infrastructure automation, plus Terraform and Git-based workflows.
- Celery Executor in production.
- CI/CD tooling (Jenkins, GitLab CI or equivalent).
- Monitoring and logging: Grafana, StatsD, ELK/Loki.
- Practical vulnerability management — CVE triage, scanning tools, patch cadence, audit closure.
- Incident, change and problem management in an enterprise/ITIL-style environment.
- Demonstrated technical leadership of a team owning production SLAs.

Good to Have
- Migration experience from legacy schedulers — Control-M, Autosys, Tidal, Rundeck, cron — onto Airflow.
- Contributions to Apache Airflow or related open-source projects.
- ServiceNow / ITSM and CMDB integration experience.
- Storage, backup (Veeam, NetBackup, Commvault), virtualisation (VMware) or network automation exposure.
- Vault, Keycloak, LDAP/Kerberos integration.
- Certifications: Astronomer Certified Airflow, CKA/CKAD, RHCE, Red Hat Ansible Automation, MySQL DBA, or a security certification.
- Regulated-industry background (BFSI, telecom, healthcare) with formal DR and audit obligations.

Pay: ₹1,800,000.00 - ₹2,000,000.00 per year

Work Location: In person

📌 Airflow Architect (Chennai)
🏢 Trizen-AI
📍 Chennai