Role Summary :

We are looking for an experienced Sr. AWS Data Engineer with 8 years of expertise in designing, building, and optimizing large-scale cloud-based data platforms. The ideal candidate should have robust hands-on experience with AWS data services, distributed computing frameworks, and real-time streaming architectures.

You will be responsible for developing scalable data pipelines, implementing modern data lake solutions, and enabling high-performance analytics using AWS and Apache Spark technologies.

Key Technical Skills :

- AWS Glue
- AWS EMR (Elastic MapReduce)
- AWS Glue ETL
- AWS Glue Data Catalog
- PySpark
- Python
- Apache Spark
- Spark Streaming
- Apache Kafka
- Apache Hudi
- Apache Iceberg
- Docker
- Amazon ECS
- Terraform
- OOPS
- Data Lake Architecture
- Real-Time Data Processing
- Distributed Data Processing
- Cloud Data Engineering

Roles &

Responsibilities :

- Design, develop, and maintain scalable data engineering solutions on AWS.
- Build and optimize ETL/ELT pipelines using AWS Glue, Glue ETL, and PySpark.
- Develop high-performance real-time streaming applications using Spark Streaming and Apache Kafka.
- Design and implement scalable data lake solutions using Apache Hudi and Apache Iceberg.
- Process and analyze high-volume, high-velocity datasets using Amazon EMR.
- Develop reusable, effective, and maintainable data processing frameworks.
- Create, optimize, and manage Glue Data Catalog for metadata management.
- Implement infrastructure automation using Terraform.
- Containerize applications using Docker and deploy workloads on Amazon ECS.
- Monitor, troubleshoot, and optimize data pipelines for performance, scalability, and reliability.




- Collaborate with data scientists, analysts, and cross-functional teams to deliver robust data platforms.
- Follow software engineering best practices including OOPS principles, code reviews, testing, and documentation.
- Ensure data quality, security, governance, and compliance across data platforms.
- Optimize Spark jobs and distributed workloads for maximum efficiency.
- Participate in architecture discussions and contribute to cloud modernization initiatives.

Required Qualifications :

- Bachelor's or Master's degree in Computer Science, Information Technology, Software Engineering, or a related field.
- 8 years of experience in Data Engineering or Big Data development.
- Robust expertise in Python, PySpark, and Apache Spark.
- Hands-on experience with AWS Glue, Glue ETL, Glue Data Catalog, and Amazon EMR.
- Strong experience building real-time streaming solutions using Apache Kafka and Spark Streaming.
- Practical knowledge of Apache Hudi and Apache Iceberg.
- Experience with Docker containers and Amazon ECS.
- Strong understanding of Terraform for Infrastructure as Code (IaC).
- Solid understanding of distributed computing concepts and cloud-native architectures.
- Excellent problem-solving and debugging skills.
- Strong communication and collaboration abilities.

Preferred Skills :

- AWS Cloud certifications.
- Certifications in Apache Spark, Kafka, Docker, or Terraform.
- Experience designing enterprise-scale data lake architectures.
- Exposure to modern DevOps and CI/CD practices.
- Knowledge of performance tuning and optimization for Spark workloads.
- Experience working in Agile/Scrum environments.

📌 Senior AWS Data Engineer (India)
🏢 Tekpillar
📍 India