- Experience in Data Pipeline Orchestration tools (Airflow, Step Functions etc)
/n /n Responsibility of / Expectations from the Role /n Design Develop and maintain Scalable Data Pipelines using Apache Spark (Scala) /n Build and Optimize ETL/ELT workflows on AWS /n Work with large scale structured and unstructured Datasets /n Develop Data Solutions using AWS services such as S3, EMR, Glue, Redshift, Athena,Lambda /n Ensure Data Security, Governance and Compliance Best Practices