Job Description: AWS PySpark Data Engineer
Job Title: AWS PySpark Data Engineer
Experience: 5+ Years
Location: Indore
Job Summary
We are looking for an experienced AWS PySpark Data Engineer with strong hands-on expertise in building scalable data engineering solutions on AWS. The role will focus on developing and optimizing ETL/ELT pipelines, processing large-scale datasets using PySpark/Spark, and building cloud-native data solutions using AWS services.
Mandatory Skills
• 5+ years of overall experience
• PySpark / Apache Spark
• Python
• SQL
• AWS Glue
• Amazon S3
• AWS EMR
• AWS Lambda
• Amazon Redshift
• ETL/ELT development
• Data Lake/Data Warehouse concepts
• Large-scale/distributed data processing
Key Responsibilities
• Design, develop and maintain scalable ETL/ELT pipelines using PySpark.
• Build data ingestion and transformation frameworks on AWS.
• Process large volumes of structured, semi-structured and unstructured data.
• Develop and optimize Spark/PySpark workloads.
• Build cloud-native Data Lake and Data Warehouse solutions.
• Implement data quality, validation, monitoring and governance.
• Monitor and troubleshoot production data pipelines.
• Participate in CI/CD, deployments and Agile delivery.
• Work closely with data architects, analytics teams and business stakeholders.