Job Summary

Build and operate AWS-native Data & AI platforms with strong data engineering and platform engineering ownership. The role focuses on AWS Glue, EMR, S3, Athena, Redshift, MWAA, Step Functions, Lambda, EventBridge, SageMaker, APIs, enterprise service integration, Git-based delivery, Data SRE, data security, Immuta/Lake Formation governed access, and AI/agentic operations for production-grade data and AI workloads.

Role AI Data Platform Engineer - AWS Experience Guide

8-11 years

Primary Skill Area

AWS Data Platforms, Glue, EMR, SageMaker, S3, Redshift, Event-Driven Data & Agentic Operations The opportunity Your key responsibilities AWS Data Engineering

- Design and build production-grade AWS data pipelines using Amazon S3, AWS Glue, PySpark, Athena, Redshift, EMR, MWAA/Airflow, Step Functions, Lambda, EventBridge, CloudWatch, IAM, and KMS.
- Develop reusable ingestion frameworks supporting batch, streaming, event-driven, CDC, API-based, file-based, database, and third-party service integration patterns.
- Build curated raw, standardised, trusted, and consumption layers using scalable lakehouse design patterns, partitioning, metadata management, and file-format optimisation.
- Optimise Spark/Glue/EMR workloads for performance, cost efficiency, scalability, and operational stability.

AWS Platform Engineering
- Create reusable AWS platform accelerators for onboarding, pipeline templates, orchestration, monitoring, reconciliation, deployment, logging, and support runbooks.
- Implement Git connectivity, branching strategy, pull requests, code reviews, CI/CD, Infrastructure as Code, controlled releases,



and environment promotion.
- Integrate AWS data platforms with enterprise APIs, source applications, messaging/event services, governance tools, security platforms, and downstream analytics consumers.
- Partner with infrastructure, IAM, network, DBA, application, and support teams to resolve connectivity, access, deployment, and production issues.
- SageMaker, AI Integration & Agentic Enablement
- Integrate AWS data platforms with Amazon SageMaker for data preparation, feature engineering, model training, deployment, MLOps workflows, and inference-ready data products.
- Support SageMaker Pipelines, Feature Store, Model Registry, Model Monitor, Bedrock where relevant, vector stores, semantic search, and RAG-ready data products.
- Apply AI-assisted and agentic operations for anomaly detection, schema drift detection, failed-job diagnosis, data quality recommendations, documentation generation, and incident summarisation.

Governance, Security & Data SRE
- Implement AWS data security controls including IAM least privilege, KMS encryption, Secrets Manager, VPC endpoints, Lake Formation, Glue Data Catalog, Macie, CloudTrail, and audit-ready access patterns.
- Integrate with Immuta, Microsoft Purview, Collibra,



enterprise IAM, monitoring platforms, data quality tools, and downstream analytics/AI consumers.
- Own Data SRE responsibilities including CloudWatch observability, SLA/SLO tracking, alerting, retry logic, restartability, root-cause analysis, incident response, and production reliability management.

Skills and attributes for success
- Core platform: AWS Glue, Amazon S3, Athena, Redshift, EMR, MWAA/Airflow, Step Functions, Lambda, EventBridge, CloudWatch.
- AI and GenAI: Amazon SageMaker, SageMaker Pipelines, Feature Store, Model Registry, Model Monitor, Bedrock optional, RAG-ready data products, Agentic AI.
- Engineering: Python, PySpark, SQL, APIs, Git, CI/CD, Shell scripting, unit testing, integration testing, data pipeline testing.
- Cloud and DevOps: Terraform/OpenTofu, CloudFormation, GitHub Actions, Azure DevOps, Jenkins, Docker, Kubernetes/EKS, policy-as-code.
- Governance and reliability: IAM, KMS, Lake Formation, Glue Data Catalog, Macie, Immuta, Purview, CloudTrail, data quality, observability, Data SRE, FinOps.

To qualify for the role, you must have
- 8-11 years of experience in data engineering, data platform operations, analytics engineering, platform engineering, or AI platform enablement.
- Solid hands-on implementation experience with cloud data platforms, APIs, Git connectivity, CI/CD, governed access patterns, SRE practices, and production operations.
- Preferred certifications aligned to the relevant cloud/platform stack, data engineering, DevOps, security, governance.

📌 EY - GDS Consulting - AI And DATA -AWS Data Engineer - Manager (Coimbatore)
🏢 EY
📍 Coimbatore