We are looking for an experienced AWS PySpark Data Engineer with robust hands-on expertise in building scalable data engineering solutions on AWS. The role will focus on developing and optimizing ETL/ELT pipelines, processing large-scale datasets using PySpark/Spark, and building cloud-native data solutions using AWS services.
Mandatory Skills
5+ years of overall experience
PySpark / Apache Spark
Python
SQL
AWS Glue
Amazon S3
AWS EMR
AWS Lambda
Amazon Redshift
ETL/ELT development
Data Lake/Data Warehouse concepts
Large-scale/distributed data processing
Key Responsibilities
Design, develop and maintain scalable ETL/ELT pipelines using PySpark .
Build data ingestion and transformation frameworks on AWS.
Process large volumes of structured, semi-structured and unstructured data.
Develop and optimize Spark/PySpark workloads.
Build cloud-native Data Lake and Data Warehouse solutions.
Implement data quality, validation, monitoring and governance.
Monitor and troubleshoot production data pipelines.
Participate in CI/CD, deployments and Agile delivery.
Work closely with data architects, analytics teams and business stakeholders.