About the Role

We are seeking a QA / Test Engineer to join our energetic and fast-paced team. This role is critical to ensuring the quality, accuracy and reliability of our multi-agent LLM product, which powers AI-driven interactions across the business.

As part of this high-impact team, the role will focus primarily on manual testing, functional testing, front-end validation, answer accuracy and source-data alignment . The successful candidate will be responsible for creating and executing detailed test cases, validating user journeys, checking AI-generated responses against the correct data sources, and ensuring the product behaves as expected across different scenarios.

They will work closely with developers, data scientists and product managers to identify defects, reproduce issues, validate fixes and support release readiness. This is a fast-paced, complex and evolving environment where strong attention to detail, structured thinking, clear documentation and practical problem-solving are essential.

Key Responsibilities

Manual Test Planning & Execution

Develop, document and execute detailed manual test plans and test cases for the AI Assistant's front-end UI, conversational flows and core product functionality.

Perform functional, regression, exploratory, smoke and usability testing across current features, enhancements and production fixes.

Validate that user journeys, prompts, responses, data retrieval and system behaviours operate as expected across different scenarios.

Create, update and manage Jira tickets for defects, test findings and follow-up actions, ensuring issues are clearly documented for the development team.

Collaborate with developers to reproduce reported issues, provide clear steps to recreate defects and validate fixes once resolved.

Maintain test evidence, test results and release validation documentation in a clear and structured way.

AI Response Validation & Accuracy Testing

Manually validate LLM-generated responses against structured and unstructured data sources to ensure accuracy, completeness and reliability.

Check that responses are grounded in the correct source data and that the AI Assistant is retrieving and presenting information appropriately.

Identify,



document and escalate hallucinations, incorrect answers, source mismatches, missing context, reasoning gaps and inconsistent outputs.

Test a range of realistic business scenarios, edge cases and user prompts to assess how the AI Assistant behaves under different conditions.

Support the development of answer validation approaches and quality checks to improve confidence in AI-generated responses.

Functional & Front-End Testing

Test front-end screens, workflows, navigation, forms, conversational interfaces and user interactions to ensure they are working correctly.

Validate that business rules, expected behaviours, permissions, access controls and user flows are functioning as intended.

Perform cross-browser, UI consistency and usability checks where required.

Support release testing by validating new functionality, regression areas and priority fixes before deployment.

Automation & Test Efficiency

Support automated testing where appropriate, but with the primary focus remaining on manual and functional test execution.

Create or maintain basic test scripts for repeatable UI, regression or performance checks where automation adds value.

Contribute to the continuous improvement of test coverage, test processes and reusable test assets.

Work with engineering teams to identify areas where automation can support efficiency, while ensuring manual validation remains solid for complex AI response testing.

Performance, Security & Access Testing

Support testing of response speed, usability and system behaviour under different workloads.

Work with security and engineering teams to validate data privacy, access control, permissions and compliance requirements in AI interactions.

Identify issues where users may receive incorrect, unauthorised or inappropriate information from the AI Assistant.

Collaboration & Reporting





Provide clear, structured defect reports and track issues from identification through to resolution.

Work closely with developers, data scientists, AI engineers, DevOps teams and product managers to refine product behaviour and improve system reliability.

Participate in agile ceremonies including stand-ups, sprint planning, retrospectives and release readiness discussions.

Communicate testing progress, risks, blockers and quality concerns clearly to the wider team.

Required Skills & Experience

Proven experience in software testing and QA methodologies, with strong hands-on experience in manual and functional testing .

Experience testing AI/LLM products, chatbots, virtual assistants or NLP-driven applications is strongly preferred.

Solid understanding of functional, regression, exploratory, smoke, usability and front-end testing techniques.

Experience creating detailed manual test cases, test scripts, test evidence and defect reports.

Ability to validate AI-generated responses against source data and identify inaccuracies, hallucinations, inconsistencies or missing context.

Basic understanding of LLMs, NLP models and AI response validation.

Strong attention to detail and a methodical approach to testing and troubleshooting.

Experience working in fast-paced environments with shifting priorities and evolving requirements.

Ability to work across multiple workstreams and prioritise testing effort based on risk, impact and urgency.

Strong written and verbal communication skills, with the ability to clearly document test results, defects and expected behaviours.

Comfortable working closely with engineering, product, data and delivery teams.

Desirable Skills

Experience using Jira or similar tools for defect tracking and test management.

Exposure to automated testing tools or frameworks, particularly for UI or regression testing.

Experience testing data-driven, API-enabled or AI-enabled products.

Experience supporting release readiness, UAT, production validation or post-release defect triage.

Familiarity with privacy, access control and security considerations in AI or data-driven applications.

📌 Manual & Functional QA Testers - AI Experience Essential (India)
🏢 Project Foundry Resourcing Services
📍 India