# Airflow / Astronomer Platform Engineer (ETL SME)

**JPID:** DLTJP00063187

**Budget:** 25–26 LPA

**Experience:** 5–8 Years

**Work Location:** Hyderabad & Bangalore

**Work Mode:** Hybrid

**Interview Mode:** Virtual

**Interview Rounds:** 2 Internal Rounds

**Work Timing:** 24/7 Rotational; once in 4 months, night shift

## Role Title

Airflow / Astronomer Platform Engineer (Contract / Short-Term Project — Migration Specialist)

## Engagement Type

Small, fixed-scope project engagement (3 phases). Estimated duration: to be scoped per phase, but structured as a short-term/contract assignment rather than a permanent hire, given the finite, milestone-driven nature of the work.

## Project Scope This Role Will Support

* **Phase 1:** Upgrade existing deployments from Astro Runtime 12.x → 13.8.0; validation, smoke testing, rollback readiness.

* **Phase 2:** Metadata database cleanup/retention (prerequisite for Airflow 3 migration).

* **Phase 3:** Migration to Airflow 3.x (Deployment Version 3.2-5), including DAG/plugin/operator/hook/API compatibility review and remediation.

* **Phase 4:** Post-upgrade validation and production readiness sign-off.

## Key Responsibilities

* Execute Astro Runtime upgrade from 12.x to 13.8.0 across multiple deployments.

* Perform DAG and application validation, smoke testing, and operational verification after each upgrade step.

* Identify, validate, and run the Airflow metadata cleanup DAG; work with application teams to define data retention windows.

* Conduct a detailed compatibility assessment of all DAGs, custom plugins, operators, hooks, APIs, and integrations against Airflow 3 requirements.

* Remediate deprecated features and breaking changes identified during the Airflow 3 compatibility review.





* Execute the migration to Deployment Version 3.2-5 (Airflow 3.x) with rollback planning at each stage.

* Coordinate testing (unit/integration/regression) with application teams prior to production cutover.

* Document upgrade steps, issues encountered, and resolutions for audit/knowledge transfer.

* Support go/no-go decisions and production readiness checks.

## Tech Stack

* **Orchestration:** Apache Airflow (2.x & 3.x), Astronomer / Astro (CLI, Runtime, Deployments)

* **OS:** Linux (RHEL/Ubuntu) — command line, shell scripting, service/process management

* **Cloud:** AWS (EC2, S3, RDS/Postgres, IAM, CloudWatch — as applicable to the setting)

* **Database:** PostgreSQL (Airflow metadata DB — cleanup, retention, performance)

* **Languages:** Python (DAGs, plugins, hooks, operators), Bash

* **Version Control:** Git/GitHub or equivalent

## Required Skills & Experience

* Hands-on experience upgrading/administering Apache Airflow (2.x, ideally exposure to 3.x) — DAG authoring, scheduler internals, operators/hooks/plugins.

* Direct experience with Astronomer / Astro platform — Astro CLI, Runtime version upgrades, Deployment management.

* Genuine, working-level Linux command-line proficiency — most upgrade, validation, rollback, and troubleshooting steps (log inspection, service checks, file/permission management,



process management) happen at the shell, so this should not be assumed as a byproduct of other skills.

* Comfortable operating in AWS-hosted environments (compute, storage, networking basics relevant to where Astro/Airflow is deployed).

* Experience troubleshooting and deploying via Kubernetes (pods, deployments, logs) since Astro runs on K8s under the hood.

* Solid Python for reviewing and remediating DAG, plugin, and hook code.

* Working knowledge of PostgreSQL for Airflow metadata cleanup, retention strategy, and performance.

* Familiarity with Airflow 2 → 3 migration specifics (breaking changes, deprecated APIs, DAG serialization changes, executor changes) — reference: Apache Airflow's official "Upgrading to Airflow 3" guide.

* Strong testing/validation discipline — smoke testing, regression testing, rollback procedures.

* Ability to work with distributed application teams to assess DAG-level impact.

## Nice to Have

* Prior experience running a similar Airflow 2→3 or major-version Airflow migration in production.

* Experience with observability tooling for Airflow (logging, metrics, alerting).

* Familiarity with enterprise change management / rollback governance processes.

## Deliverables

* Upgraded and validated Runtime 13.8.0 environments.

* Metadata cleanup completed and verified.

* DAG/plugin/operator compatibility assessment report.

* Migrated and validated Airflow 3.x (Deployment Version 3.2-5) environments.

* Post-migration validation and production readiness sign-off documentation.

Pay: Up to ₹2,600,000.00 per year

Work Location: Hybrid remote in Bangalore City, Bengaluru, Karnataka

📌 Airflow / Astronomer Platform Engineer (ETL SME) (Bengaluru)
🏢 Clock Edge Technologies Pvt Lt
📍 Bengaluru