Overall Experience - 5+
Location - Indore

Job Summary
We are seeking an experienced AWS PySpark Developer with robust expertise in building scalable data engineering solutions on AWS Cloud. The ideal candidate should have hands-on experience in PySpark, Spark, AWS data services, ETL development, and large-scale data processing. The role involves designing, developing, and optimizing cloud-native data pipelines, data lakes, and analytics solutions.

Key Responsibilities
• Design and develop scalable ETL/ELT pipelines using PySpark.
• Build and maintain data ingestion frameworks on AWS.
• Develop batch and real-time data processing solutions.
• Optimize Spark jobs for performance and cost efficiency.
• Work with structured, semi-structured, and unstructured datasets.
• Develop data lake and data warehouse solutions.
• Implement data quality checks, monitoring, and alerting.
• Collaborate with business stakeholders, architects,



and analytics teams.
• Support production deployments and troubleshoot critical issues.
• Follow DevOps and CI/CD best practices.

Mandatory Technical Skills
AWS Services
• Amazon S3
• AWS Glue
• AWS EMR
• AWS Lambda
• Amazon Redshift
• AWS IAM
• CloudWatch
• AWS Step Functions

Big Data & Data Engineering
• PySpark
• Apache Spark
• Spark SQL
• DataFrames & RDDs
• ETL/ELT Development
• Data Modeling
• Data Warehousing Concepts

Programming
• Python
• SQL
• Shell Scripting

Database Technologies
• Redshift
• PostgreSQL
• MySQL
• DynamoDB (Preferred)

DevOps & CI/CD
• Git
• Jenkins / AWS CodePipeline
• Docker
• Terraform (Preferred)

📌 AWS Pyspark Data Engineer (Indore)
🏢 Tata Consultancy Services
📍 Indore