We are looking for QA Analysts to ensure the quality, reliability, usability, and business accuracy of enterprise AI Assist solutions. The role will cover functional testing, regression testing, integration testing, user acceptance testing support, test automation, and ongoing quality monitoring.

Because the solution includes generative AI capabilities, the QA Analysts will also evaluate the relevance, consistency, safety, and accuracy of AI-generated responses.

Key Responsibilities

- Review business and functional requirements to identify test scenarios.
- Develop comprehensive test strategies, test plans, test cases, and test data.
- Perform functional, integration, regression, system, and end-to-end testing.
- Validate user journeys across AI Assist applications and integrated enterprise systems.
- Test APIs, workflows, data transformations, and system integrations.
- Support business users during user acceptance testing.
- Document, prioritise, and track defects through resolution.
- Conduct root-cause analysis for recurring quality issues.
- Build and maintain automated regression test suites.
- Automate API, user-interface, and integration testing where feasible.
- Incorporate automated tests into CI/CD pipelines.
- Validate the accuracy, relevance, completeness, and consistency of AI-generated responses.
- Test prompts, context retrieval, grounding, citations, fallback behaviour, and escalation workflows.
- Evaluate AI responses across different user profiles, queries, data conditions, and edge cases.
- Test for hallucinations, unsafe responses, bias, sensitive-data exposure, and policy violations.
- Establish repeatable evaluation datasets and expected-answer benchmarks.
- Support ongoing monitoring of production quality and AI-response performance.
- Track test coverage, defect leakage,



pass rates, response quality, and release readiness.
- Work with developers and business analysts to improve testability and acceptance criteria.
- Prepare quality reports and release recommendations.

Required Qualifications
- Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field.
- 3–7 years of experience in software quality assurance or testing.
- Experience with functional, integration, regression, system, and UAT testing.
- Strong understanding of software testing lifecycle and defect-management processes.
- Experience testing APIs using tools such as Postman, REST Assured, SoapUI, or equivalent.
- Experience with test-management and defect-tracking tools such as Jira, Azure DevOps, TestRail, or equivalent.
- Strong analytical skills and attention to detail.
- Ability to translate requirements into clear and comprehensive test scenarios.
- Solid written communication and documentation skills.
- Experience working in Agile delivery environments.

Preferred Qualifications
- Experience with automation tools such as Selenium, Cypress, Playwright, Appium, or equivalent.
- Experience with API and performance-test automation.
- Familiarity with CI/CD pipelines and automated quality gates.
- Experience testing AI, machine-learning, chatbot, conversational AI, or generative AI solutions.
- Familiarity with LLM evaluation concepts such as groundedness, relevance, hallucination, toxicity, bias, and safety.
- Experience with AI evaluation or observability tools.
- Familiarity with Python, JavaScript, Java, or another scripting language.
- Experience setting up synthetic monitoring, production checks, or continuous quality monitoring.
- Exposure to contact-centre, customer-service, CRM, or knowledge-management platforms.

📌 QA Analyst – AI Assist and Test Automation (India)
🏢 Spydra
📍 India