Senior Test Lead

Pune, Maharashtra

Job Summary

Design and execute testing strategies for AI/LLM systems with solid grounding in LLM testing concepts.

Validate and evaluate LLM outputs using frameworks such as DeepEval, RAGAS, TruLens, and LangSmith.

Perform prompt validation, optimization, and robustness testing across diverse use cases.

Detect, analyze, and mitigate hallucinations to ensure response accuracy and reliability.

Assess quality, bias, and performance of LLM-driven applications in production-like scenarios.

Support validation of multi-agent AI systems and contribute to continuous quality improvements.

Key Responsibilities

Define and execute test strategies for AI/LLM‑based applications.

Validate LLM responses using evaluation frameworks (DeepEval, RAGAS, TruLens, LangSmith).

Perform prompt validation, robustness testing,



and regression testing for LLM workflows.

Identify, analyze, and mitigate hallucinations, bias, and response inconsistencies.

Ensure quality, accuracy, and reliability of LLM outputs across business scenarios.

Support testing and validation of multi‑agent AI systems where applicable.

Skill Requirements

Skill

Mandatory

Proficiency Level (1-5)

AI/LLM Testing concepts

Must Have

4–5

LLM evaluation frameworks (DeepEval, RAGAS, TruLens, LangSmith),

Must Have

4

Prompt validation & testing

Must Have

4–5

Hallucination detection

Must Have

4

Multi-agent system validation

Good to Have

3–4

📌 Senior Test Lead (Pune)
🏢 HCLTech
📍 Pune