GenAI Testing skilled with experience in validating Large Language Models (LLMs), Agentic AI systems, RAG applications, and AI-powered business solutions. The candidate will drive quality assurance for GenAI implementations, ensuring accuracy, reliability, security, and business alignment of AI-generated outcomes.

Mandatory Skills

GenAI Testing Concepts

LLM Validation & Evaluation

Prompt Engineering

RAG Testing

Functional & Automation Testing

API Testing

Python

AI Model Evaluation Techniques

Jira / Azure Dev Ops

Agile Delivery Model

Key Responsibilities

Design and execute test strategies for GenAI, RAG, Copilot, and Agentic AI applications.

Validate AI responses for accuracy, relevance, consistency, bias, toxicity, and hallucinations.

Develop AI-specific test scenarios, prompt libraries, and evaluation frameworks.





Create automated validation approaches for LLM outputs.

Perform prompt engineering and prompt optimization activities.

Test Retrieval-Augmented Generation (RAG) workflows and knowledge-grounding mechanisms.

Validate AI guardrails, compliance requirements, and security controls.

Collaborate with Product Owners, Business SMEs, Data Scientists, and AI Architects.

Define KPIs and quality metrics for AI solution evaluation.

Support GenAI adoption initiatives and recommend QA best practices.

Good to Have

Azure OpenAI / OpenAI

Lang Chain / Lang Graph

MCP (Model Context Protocol)

Agentic AI Platforms

Git Hub Copilot

AI Security Testing

Python-based Automation Frameworks

📌 GenAI QA / Testing Engineer (India)
🏢 TRIGENT SOFTWARE PRIVATE
📍 India