We are seeking an experienced Senior Data Platform Engineer with strong expertise in Apache Airflow, AWS Cloud, Database Administration, and Data Pipeline

Operations. The ideal candidate will be responsible for deploying, administering,

maintaining, and troubleshooting enterprise-scale data platforms hosted across both on-premises infrastructure and AWS Cloud environments. The role requires hands-

on expertise in Airflow platform management, PostgreSQL, Amazon Redshift,

Snowflake, Linux administration, and data pipeline support.

Key Responsibilities

Apache Airflow Administration & Platform Engineering

 Build, deploy, configure, and maintain Apache Airflow environments across on-

premises infrastructure and AWS Cloud.

 Perform Airflow platform administration activities including start/stop operations,

upgrades, patching, configuration management, and capacity planning.

 Design, deploy, and manage Apache Airflow and Amazon Managed Workflows for Apache Airflow (MWAA) environments.

 Provision and manage AWS infrastructure required for Airflow using AWS

CloudFormation templates and Infrastructure as Code (IaC) practices.

 Automate Airflow installation, configuration, and maintenance using Shell scripting on Linux/UNIX platforms.

 Manage Airflow DAG deployments through Jenkins CI/CD pipelines and release management processes.

 Monitor Airflow schedulers, workers, executors, metadata databases, and DAG performance to ensure platform stability and reliability.

 Troubleshoot and resolve Airflow platform, scheduler, executor, and DAG-related issues across development, UAT, and production environments.

 Manage Airflow metadata databases and perform performance tuning activities.

 Administer Airflow hosted on Linux/UNIX servers, ensuring platform security,

performance, and availability.

 Ability to develop, maintain, and troubleshoot Python-based Airflow DAGs and workflow dependencies.

 Support High Availability (HA), Disaster Recovery (DR), backup, restore, and failover testing activities

AWS Cloud & Data Platform Management

 Design and manage MWAA clusters, including workplace configuration,

scaling, networking, security, and performance optimization.

 Manage AWS services supporting Airflow and data platforms, including EC2,

RDS PostgreSQL, ELS, Lambda, Redshift, S3, IAM,



CloudWatch, VPC, Load

Balancers, and Auto Scaling services.

 Optimize cloud infrastructure utilization while ensuring high availability, security,

and operational efficiency.

Data Pipeline Operations & Support

 Monitor, support, and maintain business-critical ETL and data pipelines running on Airflow.

 Investigate and resolve failed DAGs, performance bottlenecks, and data processing issues.

 Perform root cause analysis for recurring pipeline failures and implement preventive measures.

 Collaborate with development and data engineering teams to improve pipeline reliability and performance.

 Ensure data platform availability and adherence to SLA requirements.

Database Administration & Support

 Administer and support PostgreSQL databases hosted on AWS.

 Manage Amazon Redshift clusters, including monitoring, maintenance,

performance tuning, and capacity management.

 Support Snowflake platform operations and coordinate with application teams for issue resolution.

 Perform database query analysis, optimization, and performance tuning.

 Administer and support Amazon RDS databases, primarily PostgreSQL, in production, DR, UAT, and development environments.

 Monitor database health, storage utilization, and system performance.

CI/CD & Automation

 Design and maintain Jenkins pipelines for Airflow DAG deployment and platform automation.

 Develop and maintain Shell scripts for Airflow installation, maintenance,

monitoring, and operational tasks.

 Implement automation solutions for platform provisioning, application deployment, patching, and operational support.

Security & Compliance

 Renew, install, and manage SSL/TLS certificates for Airflow platforms and associated services.

 Support security vulnerability remediation and compliance initiatives.

 Ensure secure authentication, access management,



and encryption standards are maintained across data platforms.

Troubleshooting & Incident Management

 Analyze and resolve complex issues related to Airflow, databases, Linux servers,

and data pipelines.

 Participate in incident management, problem management, and change management processes.

 Develop and maintain SOPs, operational runbooks, and technical documentation.

 Provide on-call support for critical production incidents and platform outages.

Linux/UNIX Administration

 Manage Linux/UNIX servers hosting Airflow and related platform components.

 Perform system monitoring, log analysis, troubleshooting, performance tuning,

and maintenance activities.

 Support platform patching, system upgrades, and infrastructure hardening.

Required Skills

 Strong expertise in Apache Airflow administration and troubleshooting.

 Experience deploying and managing Airflow on AWS and on-premises servers.

 Hands-on experience with PostgreSQL and Amazon Redshift administration.

 Working knowledge of Snowflake platform operations and support.

 Strong SQL development, query analysis, and performance tuning skills.

 Experience supporting enterprise ETL/Data Pipeline environments.

 Strong Linux/UNIX system administration experience.

 Knowledge of SSL/TLS certificate management and renewal processes.

 Experience with AWS services such as EC2, RDS, S3, IAM, CloudWatch, and networking fundamentals.

 Strong troubleshooting, problem-solving, and analytical skills.

Preferred Skills

 Experience with Python scripting and Airflow DAG development.

 Knowledge of CI/CD tools such as Jenkins, GitHub, and

CloudFormation/Terraform.

 Experience with monitoring tools such as Splunk, Grafana, Dynatrace, or

CloudWatch.

 Familiarity with Data Engineering, Data Warehousing, and Cloud Platform

Operations.

 Experience supporting production environments with stringent SLA requirements.

Experience

 7 to 12 years of overall IT experience.

 3+ years of hands-on experience with Apache Airflow administration.

 Experience supporting mission-critical data platforms in cloud and hybrid environments.

Education

 Bachelor's or Master's degree in Computer Science, Information Technology,

Engineering, or related field.

📌 Senior Data Platform Engineer (India)
🏢 Principle Pride
📍 India