- Design develop and optimize largescale data processing pipelines using Apache Spark
- Build and maintain ETLELT workflows using AWS Glue Amazon S3 and Amazon Athena
- Orchestrate endtoend data pipelines and processing workflows using AWS Step Functions
- Develop ingestion frameworks to handle multivendor data schemas including Ericsson and Oracle datasets
- Standardize transform map and integrate heterogeneous vendor data into common data models
- Implement data validation reconciliation error handling and dataquality controls across ingestion pipelines
- Optimize Spark jobs Glue workloads storage structures queries and underlying datastore performance
- Monitor pipeline execution troubleshoot failures perform rootcause analysis RCA and improve processing reliability
- Support schema evolution metadata management partitioning and performance optimization for large datasets
- Collaborate with architecture platform analytics and downstream application teams to ensure scalable and reliable data delivery
Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.