We are looking for an 6+years AWS Data Engineer with solid hands-on expertise in building, developing, and optimizing scalable data pipelines on AWS. The candidate should have practical experience with data ingestion, transformation, orchestration, performance tuning, and production troubleshooting.

Key Responsibilities

- Design and develop end-to-end AWS data pipelines using S3, Glue, Lambda, and Step Functions.
- Implement data ingestion from files, databases, APIs, and event-driven sources.
- Build data validation, cleansing, transformation, and enrichment workflows using Python, SQL, and PySpark.
- Optimize Athena queries through partitioning, Parquet/ORC formats, compression, and cost-effective data scanning.
- Apply Spark optimization techniques, including broadcast joins, data skew handling, and salting.
- Implement S3 storage lifecycle policies and optimize storage costs.
- Configure workflow retries, exception handling, idempotency, and failure recovery.
- Monitor production pipelines, troubleshoot failures,



and ensure data quality, reliability, and performance.
- Maintain technical documentation and support production deployments.

Required Skills
- Strong hands-on experience with AWS services: S3, Glue, Lambda, Step Functions, and Athena.
- Proficiency in Python, SQL, and PySpark.
- Experience designing end-to-end ETL/ELT pipelines and data lake architectures.
- Strong understanding of S3 raw, curated, and consumption layers, partitioning, and storage formats.
- Experience with Spark performance optimization, broadcast joins, and data skew/salting.
- Experience of AWS Step Functions Standard and Express workflows, retries, error handling, and orchestration.
- Experience with production monitoring, troubleshooting, data validation, and pipeline recovery.
- Strong understanding of data processing trade-offs, scalability, and cost optimization.

📌 AWS Data Engineer (Chandigarh)
🏢 Trantor
📍 Chandigarh