Role Overview

We are seeking a highly skilled Test Automation Engineer to join the Agentic AI Squad within our AI Platform organization. The ideal candidate will have strong experience in designing, implementing, and maintaining automated test frameworks for complex distributed systems, with particular focus on AI/LLM-powered capabilities, agentic workflows, and high-scale backend services.

This role involves building robust automation pipelines, validating model-driven behaviors, ensuring system reliability through automated quality gates, and collaborating closely with AI researchers, backend engineers, and product teams to ensure end-to-end quality across the platform.

Key Responsibilities

Automation & Quality Engineering
Design, implement, and maintain scalable automated test suites for API, integration, end-to-end, and workflow-level testing of agentic AI systems.
Build automation frameworks that validate correctness, robustness, safety, and determinism in LLM-driven behaviors and multi-agent orchestration.
Develop synthetic datasets, fixtures,



and mocks to replicate complex agentic scenarios and edge cases.

AI/LLM-Specific Testing
Implement automated evaluation pipelines for LLM outputs, including functional tests, regression checks, safety tests, and model-behavior verification.
Collaborate with ML Engineers to integrate model evaluation metrics (e.g., hallucination detection, grounding accuracy, response consistency) into CI/CD systems.
Validate prompt templates, agent tools, retrieval workflows, and chain-of-thought instrumentation.

Platform Integration
Work with backend and platform teams to ensure test coverage across microservices, feature flags, workflow orchestrators, and event-driven systems.
Contribute to reliable system design and ensure recent features include explicit acceptance criteria and automation strategies.

CI/CD & Release Quality
Integrate automated testing pipelines into GitHub Actions (or equivalent), ensuring f

📌 Ai Test Automation Engineer Noida (India)
🏢 Topcoder
📍 India