Description:

Position :- Airflow Infrastructure Engineer
Experience :- 5+ Year relevant in Airflow & Docker
Rate Card :- Rs 1070/hr - MAX
Shift Timing : 11:00 AM to 08:00 PM IST
Location :- Global Village, Bangalore / Gurgaon NTT Office (Minimum 2 Days Work from Office Non Negotiable)

This is not a DevOps position & does not involve working in any cloud ecosystem (AWS / Azure /GCP). Set the expectation accordingly with

MUST Have Mandatory skills (non-negotiable)
5 Years experience with Airflow Administration & must have hands on experience with Airflow DAG.
5 Years of hands on experience with Docker & should have experience completely owning Docker image and
Docker Compose configuration.
Must have experience with Control-M for scheduling & automation.
experience with DBT & Informatica will be a major plus but not mandatoy

Required Skills & Experience
5+ years of hands-on experience with Docker and Docker Compose in enterprise or production environments, including multi-service deployments on Linux VMs.
Demonstrated experience administering Apache Airflow scheduler, webserver, workers, metadata database, connections, and configuration management.
Strong Linux systems administration skills systemd services, file permissions, environment variables, log management, and VM-level troubleshooting.
Experience troubleshooting containerized application issues, including image compatibility problems, networking failures, volume mount errors, and startup sequencing.
Experience managing Python library dependencies within Airflow environments, including package installation, version conflicts, and virtual environment or Docker-layer management.
Experience with dbt project structure and dbt CLI execution within Airflow-orchestrated pipelines.
Experience with NAS-based deployment patterns, including shared file system mounts within Docker containers.
Experience with GitLab CI/CD pipelines in a data platform or ETL modernization context, particularly for DAG and artifact deployment workflows.
Experience integrating Airflow with enterprise scheduling tools such as BMC Control-M, including REST API-based job triggering and authentication configuration.
Experience working in multi-environment deployment architectures (Development, QA, PreProd, Production) with structured promotion and change control processes.
Strong written and verbal communication skills able to interface effectively with infrastructure/platform teams and delivery-focused data engineering teams.
Experience producing clear operational documentation, runbooks, and infrastructure diagrams.

Preferred
Experience supporting large-scale data migration programs with strict environment stability requirements.
Experience with secrets management tooling (HashiCorp Vault, CyberArk, or equivalent) for securing pipeline credentials.

Role Overview
The Airflow Infrastructure Engineer is a dedicated hands-on specialist responsible for owning the Docker-based deployment, setting configuration, and operational reliability of the Airflow platform across Development, QA, PreProd, and Prod environments. This role exists to protect delivery velocity by ensuring that data engineers can focus on Informatica-to-dbt conversion rather than troubleshooting infrastructure.





The Airflow Infrastructure Engineer owns the full lifecycle of the containerized Airflow environment from resolving Docker image compatibility issues to ensuring stable, reproducible Docker Compose deployments on VMs. Our Airflow DAGs orchestrate both Python EL scripts and dbt transformation scripts; when new libraries need to be installed, or when database connectivity issues arise within a DAG, the resolution of those issues is this role s responsibility.

This role also owns the configuration work required to support Control-M integration. Control-M triggers Airflow via REST API calls; the Airflow Infrastructure Engineer is responsible for configuring and maintaining the Airflow REST API endpoint, managing authentication and access controls, validating that Control-M job triggers are correctly received and processed by Airflow, and troubleshooting any failures at the Control-M-to-Airflow boundary.

Key Responsibilities

Docker & Environment Ownership
Own the Docker image and Docker Compose configuration across all environments (Development, QA, PreProd, and Production) including image selection, versioning, and Compose service definitions ensuring consistency and reproducibility across our VM-based deployment architecture.
Validate, troubleshoot, and resolve compatibility issues with infrastructure-provided Docker images for Airflow, ensuring they function correctly with our dbt, Python EL, and supporting service configurations.
Manage container networking, volume mounts, and setting variable configurations across deployment tiers.
Define and enforce a structured environment promotion process ensuring changes are validated in lower environments before being promoted to QA, PreProd and Production.
Establish and document rollback procedures for failed deployments.

Airflow Platform Administration
Administer and maintain the Airflow environment across all deployment tiers, including scheduler, webserver, worker, and metadata database components.
Manage Airflow configuration (airflow.cfg, environment variables, connections, variables, and pools) in alignment with the program s pipeline requirements across both migration workstreams.
Manage Python library dependencies within the Airflow environment installing, upgrading, and validating packages required by DAGs that invoke dbt and Python EL scripts.
Troubleshoot and resolve database connectivity issues arising from DAG execution, including connection pool configuration, driver compatibility, and credential management.
Monitor Airflow scheduler health, task execution, and log output; proactively identify and resolve platform-level issues before they impact delivery.
Support DAG deployment workflows in coordination with the CI/CD pipeline, ensuring DAG files are correctly promoted to the shared NAS and picked up by the Airflow scheduler.
Establish and maintain baseline alerting for Airflow scheduler failures,



task SLA breaches, and environment health thresholds.

Airflow Integration with Control-M
Configure and maintain the Airflow REST API endpoint used by Control-M to trigger DAG runs including authentication setup, access controls, and endpoint validation across environments.
Validate end-to-end job triggering: confirm that Control-M REST API calls are correctly received, authenticated, and processed by Airflow, and that DAG execution initiates as expected.
Troubleshoot failures at the Control-M-to-Airflow boundary, distinguishing between orchestration-layer issues and Airflow platform issues, and coordinating resolution with the relevant teams.
Coordinate with the scheduling team on Control-M job definitions, environment cutovers, scheduling changes, and production deployment windows.
Maintain documentation of the Control-M integration configuration, including API endpoint details, authentication patterns, and known failure modes.

Infrastructure Collaboration & Image Integration
Serve as the primary point of contact between the delivery team and infrastructure/platform teams for all workplace-related requests, image issues, and dependency resolution.
Systematically diagnose and resolve compatibility issues between platform-provided Docker images and our specific Airflow + dbt + Python EL stack replacing trial-and-error with structured root cause analysis.
Communicate environment requirements, dependency constraints, and change requests to platform teams with sufficient technical precision to minimize iteration cycles.
Maintain a written log of image issues, resolutions, and configuration decisions to support knowledge transfer and reduce repeated effort.

Documentation & Operational Readiness
Produce and maintain infrastructure runbooks covering Docker image and Compose configurations, environment setup procedures, common failure scenarios, and recovery steps.
Document the environment architecture across all deployment tiers, including VM layout, Docker service topology, NAS mount points, and network configuration.
Support production readiness reviews and cutover planning from an infrastructure perspective.
Contribute to knowledge transfer activities to ensure the operations team can sustain the platform post-program.

Program Context
This role operates within a large-scale data platform modernization program migrating from Informatica PowerCenter to a Python, dbt, and Airflow-based architecture. Our tech stack consists of Airflow + dbt + Python EL scripts, deployed on VMs via Docker Compose, scheduled by Control-M. We have two parallel delivery workstreams with a shared infrastructure layer that this role is responsible for stabilizing and operating.

The Airflow Infrastructure Engineer will work closely with the CI/CD Engineer, Technical Leads, and infrastructure/platform teams. This is a hands-on execution role, not an advisory one the program requires someone who will own setting problems end-to-end and resolve them with urgency.

Disclaimer : This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

📌 Airflow Infrastructure Engineer (Panipat)
🏢 SSD Shared Services
📍 Panipat