- Solid Expertise in AWS Data Engineering
- Hands-on expertise in Spark and Scala
- Extensive experience with AWS Cloud Services (S3, EMR, Glue, Redshift, Athena etc)
- Solid understanding of Distributed computing
- Proficiency in SQL
Experience in Data Pipeline Orchestration tools (Airflow, Step Functions etc)
Responsibility of / Expectations from the Role
- Design Develop and maintain Scalable Data Pipelines using Apache Spark (Scala)
- Build and Optimize ETL/ELT workflows on AWS
- Work with large scale structured and unstructured Datasets
- Develop Data Solutions using AWS services such as S3, EMR, Glue, Redshift, Athena,Lambda
- Ensure Data Security, Governance and Compliance Best Practices