Quality Assurance (QA) Engineer – Conversational AI & Agentic AI
Job Overview
We are looking for a Quality Assurance (QA) Engineer with 3-7 years of experience in testing, QA, and reporting, specifically within the field of Conversational AI and Agentic AI systems. The ideal candidate will have a robust understanding of standard testing practices and be passionate about ensuring the highest quality standards in AI-driven applications. This role involves validating conversational experiences, AI-powered workflows, autonomous agent behaviors, integrations with external tools and APIs, and ensuring the reliability, safety, and performance of AI systems in production environments.
Key Responsibilities
• Develop and execute test cases, scripts, and procedures for Conversational AI and Agentic AI applications.
• Collaborate with product managers, developers, data scientists, prompt engineers, and other stakeholders to ensure quality standards are met throughout the development lifecycle.
• Perform functional, regression, integration, performance,
and end-to-end testing across conversational AI platforms and agent-based AI systems.
• Validate AI agent decision-making, task execution, multi-step workflows, reasoning paths, and interactions with external tools, APIs, and knowledge sources.
• Test LLM-powered applications for accuracy, relevance, consistency, hallucinations, guardrail compliance, and response quality.
• Evaluate agent orchestration capabilities, including multi-agent collaboration, task decomposition, context management, memory retention, and tool utilization.
• Identify, document, and track defects using JIRA and provide detailed reports on testing results.
• Analyze requirements and design comprehensive test strategies covering conversational flows, autonomous agent behaviors, integrations, and edge-case scenarios.
• Conduct prompt, response, and workflow validation to ensure optimal performance and user experience.
• Perform security, privacy,
📌 Functional Leader (India)
🏢 EXL
📍 India