About the Role

We are looking for an
AI QA Engineer
to help ensure the quality, accuracy, reliability and performance of an enterprise AI platform used by large
Public Sector Undertakings (PSUs)
for tender and procurement evaluation.

The platform uses
Generative AI, Agentic AI, Document AI and open-source models running on NVIDIA GPU infrastructure
to automate complex, document-heavy procurement workflows.

This is not a traditional software QA role. You will work at the intersection of
AI evaluation, software testing and enterprise product quality
, testing systems where outputs can be probabilistic and where correctness depends on context, reasoning and source documents.

What You Will Work On
- Design and execute
test strategies for AI-powered document workflows
.
- Build test datasets and evaluation benchmarks for tender documents, technical specifications, eligibility criteria, commercial conditions and other procurement content.
- Evaluate AI outputs for
accuracy, completeness, consistency, relevance and hallucination
.
- Test
LLM, VLM, RAG and Agentic AI workflows




across different document types and scenarios.
- Validate whether AI-generated answers and extracted information are correctly grounded in source documents.
- Develop automated evaluation pipelines to measure AI quality at scale.
- Perform
regression testing
whenever models, prompts, agents, retrieval pipelines or application components are changed.
- Test AI agents for correct
planning, tool usage, decision-making, workflow execution and failure handling
.
- Identify edge cases involving ambiguous, incomplete, conflicting or poorly formatted procurement documents.
- Benchmark different AI models and configurations based on
accuracy, latency, reliability and cost
.
- Test application APIs, backend services and end-to-end workflows.
- Work with AI engineers to identify root causes of model and pipeline failures.
- Track and analyze production issues and convert them into repeatable test cases.
- Es

📌 Quality Assurance Engineer (Noida)
🏢 Valiance Solutions
📍 Noida