Software Development Engineer in Test (SDET) / Automation Engineer The SDET will own automation engineering for GTS data platforms and build reusable frameworks capable of validating Spark workloads, orchestration pipelines, cloud integrations, and data delivery processes operating across GCP and AWS.
This role is highly technical and combines software engineering, cloud technologies, data engineering,
and automation expertise.
Key Responsibilities
Design and develop enterprise-grade automation frameworks for GTS platforms.
Build reusable libraries for validating Spark transformations and data pipelines.
Automate Airflow DAG validation and orchestration testing.
Develop regression suites covering:
o Data quality o Data transformations o Workflow execution o Failure handling and recovery
Create automated comparison frameworks between source and target systems.
Implement performance, stability, and volume testing for Spark applications.
Automate test execution within CI/CD pipelines.
Enable nightly regression execution and automated reporting.
Work closely with Engineering teams to shift testing left and improve delivery quality.