This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Quality Assurance Engineer based in India.

This is a high-ownership QA role focused on ensuring the quality, reliability, and safety of AI-powered products in a fast-moving engineering environment. You will design and scale automated testing across conversational AI, LLMs, RAG pipelines, voice technologies, APIs, and user interfaces. Your work will directly influence whether new releases are ready for customers, with a strong focus on evidence-based quality decisions. You will build and strengthen testing frameworks, evaluation harnesses, and CI/CD quality gates from the ground up. The role combines traditional software quality practices with the unique challenges of testing non-deterministic AI systems. You will collaborate closely with Engineering, Product, and QA teams while taking ownership of emerging quality challenges. This is an ideal prospect for an experienced QA automation professional who wants to play a key role in shaping AI product quality at scale.





n

Accountabilities

•
Build, maintain, and expand automated testing for AI-powered voice and chat agents, covering individual conversational turns through complete roleplay and end-to-end flows.

•
Design and execute tests for LLM output quality, including correctness, consistency, structured outputs, language fidelity, prompt regressions, and other non-deterministic behaviors.

•
Develop evaluation approaches using LLM-as-a-judge techniques, golden datasets, tolerance-based assertions, and other AI evaluation methodologies.

•
Validate RAG pipelines, with particular attention to retrieval relevance, grounding, faithfulness, and overall answer quality.

•
Test voice pipelines, including speech-to-text and text-to-speech accuracy, multilingual behavior, and real-time, low-latency performance.

•
Automate testing across UI/E2E, API, database, AI-output, desktop, and mobile

📌 Quality Assurance Engineer (India)
🏢 Jobgether
📍 India