We are seeking an experienced AWS PySpark Developer with robust expertise in building scalable data engineering solutions on AWS Cloud. The ideal candidate should have hands-on experience in PySpark, Spark, AWS data services, ETL development, and large-scale data processing. The role involves designing, developing, and optimizing cloud-native data pipelines, data lakes, and analytics solutions.
Key Responsibilities
Design and develop scalable ETL/ELT pipelines using PySpark.
Build and maintain data ingestion frameworks on AWS.
Develop batch and real-time data processing solutions.
Optimize Spark jobs for performance and cost efficiency.
Work with structured, semi-structured, and unstructured datasets.
Develop data lake and data warehouse solutions.
Implement data quality checks, monitoring, and alerting.
Collaborate with business stakeholders, architects, and analytics teams.
Support production deployments and troubleshoot critical issues.
Follow DevOps and CI/CD best practices.