Key Responsibilities
Design, develop, and maintain data ingestion, transformation, and integration pipelines using Python, PySpark, and SQL. Build scalable ETL/ELT workflows for structured and unstructured data across AWS data ecosystems. Implement best practices in data quality, performance optimization, and pipeline automation. Collaborate with architects and analysts to define data models, schemas, and integration frameworks. Work with cloud-native services such as: AWS: Glue, Redshift, S3, Lambda, EMR,Databricks
• Develop reusable components and frameworks for pipeline orchestration and monitoring.