The Data Engineer builds, optimises, and maintains scalable data pipelines across Aviva's cloud and big data ecosystems. Key Responsibilities
- Build ETL/ELT pipelines in Python, PySpark and Airflow. - Implement AWS data patterns: S3, Glue, Lambda, EMR, Kinesis, Redshift. - Deliver scalable, production grade transformations. - Ensure data quality, lineage, validation and monitoring. - Collaborate with architects, analysts, data scientists. Skills & Experience
- Strong Python (Pandas), PySpark and Airflow DAG creation. - Large scale data engineering experience. - AWS engineering experience or cloud equivalents. - Robust SQL and data modelling. Nice to Have