We are looking for an 6+years AWS Data Engineer with strong hands-on expertise in building, developing, and optimizing scalable data pipelines on AWS. The candidate should have practical experience with data ingestion, transformation, orchestration, performance tuning, and production troubleshooting.
Key Responsibilities
• Design and develop end-to-end AWS data pipelines using S3, Glue, Lambda, and Step Functions.
• Implement data ingestion from files, databases, APIs, and event-driven sources.
• Build data validation, cleansing, transformation, and enrichment workflows using Python, SQL, and PySpark.
• Optimize Athena queries through partitioning, Parquet/ORC formats, compression, and cost-effective data scanning.
• Apply Spark optimization techniques, including broadcast joins, data skew handling, and salting.
• Implement S3 storage lifecycle policies and optimize storage costs.
• Configure workflow retries, exception handling, idempotency, and failure recovery.
• Monitor production pipelines, troubleshoot failures,



and ensure data quality, reliability, and performance.
• Maintain technical documentation and support production deployments.

Required Skills
• Strong hands-on experience with AWS services: S3, Glue, Lambda, Step Functions, and Athena.
• Proficiency in Python, SQL, and PySpark.
• Experience designing end-to-end ETL/ELT pipelines and data lake architectures.
• Robust understanding of S3 raw, curated, and consumption layers, partitioning, and storage formats.
• Experience with Spark performance optimization, broadcast joins, and data skew/salting.
• Experience of AWS Step Functions Standard and Express workflows, retries, error handling, and orchestration.
• Experience with production monitoring, troubleshooting, data validation, and pipeline recovery.
• Strong understanding of data processing trade-offs, scalability, and cost optimization.

📌 AWS Data Engineer (Chandigarh)
🏢 Trantor
📍 Chandigarh