We are seeking a hands-on AI Engineer to design and execute end-to-end testing strategies for agentic AI solutions, including multi-agent systems in production-grade environments. This role partners with the Agentic Operations Team to ensure resiliency, reliability, accuracy, latency, orchestration correctness, and scale. You will establish QA frameworks, build reusable test artifacts, drive macro-level validations across complex workflows, and lead the QA function for Agentic AI from Dev to Prod.

Key Responsibilities
- Agentic & MultiAgent Testing
- Reliability,Resiliency, and Latency
- Accuracy & Macro-Level Validations
- Scale & Orchestration
- Dev Prod Readiness
- Define and own the QA strategy for agentic/multi-agent AI systems across dev, staging, and prod.
- Mentor a team of QA engineers; establish testing standards, coding guidelines for test harnesses, and review practices.
- Partner with Agentic Operations, Data Science, MLOps, and Platform teams to embed QA in the SDLC and incident response.
- Design tests for agent orchestration, tool calling, planner-executor loops,



and inter-agent coordination (e.g., task decomposition, handoff integrity, and convergence to goals).
- Validate state management, context windows, memory/knowledge stores, and prompt/graph correctness under varying conditions.
- Implement scenario fuzzing (e.g., adversarial inputs, prompt perturbations, tool latency spikes, degraded APIs).
- Create resilience testing suites: chaos experiments, failover, retries/backoff, circuit-breaking, and degraded mode behavior.
- Establish latency SLOs and measure end-to-end response times across orchestration layers (LLM calls, tool invocations, queues).
- Ensure reliability through soak tests, canary verifications, and automated rollbacks.
- Define ground-truth and reference pipelines for task accuracy (exact match, semantic similarity, factuality checks).
- Build macro validation frameworks that validate task outcomes across multi-step agent workflo

📌 AI Test Engineer (Hyderabad)
🏢 Qentelli
📍 Hyderabad