Overall Experience - 5+

Location - Indore

Job Summary

We are seeking an experienced AWS PySpark Developer with robust expertise in building scalable data engineering solutions on AWS Cloud. The ideal candidate should have hands-on experience in PySpark, Spark, AWS data services, ETL development, and large-scale data processing. The role involves designing, developing, and optimizing cloud-native data pipelines, data lakes, and analytics solutions.

Key Responsibilities

- Design and develop scalable ETL/ELT pipelines using PySpark.
- Build and maintain data ingestion frameworks on AWS.
- Develop batch and real-time data processing solutions.
- Optimize Spark jobs for performance and cost efficiency.
- Work with structured, semi-structured, and unstructured datasets.
- Develop data lake and data warehouse solutions.
- Implement data quality checks, monitoring, and alerting.




- Collaborate with business stakeholders, architects, and analytics teams.
- Support production deployments and troubleshoot critical issues.
- Follow DevOps and CI/CD best practices.

Mandatory Technical Skills

AWS Services

- Amazon S3
- AWS Glue
- AWS EMR
- AWS Lambda
- Amazon Redshift
- AWS IAM
- CloudWatch
- AWS Step Functions

Big Data & Data Engineering

- PySpark
- Apache Spark
- Spark SQL
- DataFrames & RDDs
- ETL/ELT Development
- Data Modeling
- Data Warehousing Concepts

Programming

- Python
- SQL
- Shell Scripting

Database Technologies

- Redshift
- PostgreSQL
- MySQL
- DynamoDB (Preferred)

DevOps & CI/CD

- Git
- Jenkins / AWS CodePipeline
- Docker
- Terraform (Preferred)

📌 AWS Pyspark Data Engineer (Indore)
🏢 Tata Consultancy Services
📍 Indore