Senior Test Lead

Experience: Not Available to Not Available years

Location: Pune, India

Skills: AI/LLM Testing concepts, LLM evaluation frameworks, DeepEval, RAGAS, TruLens, LangSmith, Prompt validation, Robustness testing, Hallucination detection, Multi-agent system validation

Job Summary
Design and execute testing strategies for AI/LLM systems with strong grounding in LLM testing concepts. Validate and evaluate LLM outputs using frameworks such as DeepEval, RAGAS, TruLens, and LangSmith. Perform prompt validation, optimization, and robustness testing across diverse use cases. Detect, analyze, and mitigate hallucinations to ensure response accuracy and reliability. Assess quality, bias, and performance of LLM-driven applications in production-like scenarios. Support validation of multi-agent AI systems and contribute to continuous quality improvements.

Key Responsibilities
Define and execute test strategies for AI/LLM‑based applications.




Validate LLM responses using evaluation frameworks (DeepEval, RAGAS, TruLens, LangSmith).
Perform prompt validation, robustness testing, and regression testing for LLM workflows.
Identify, analyze, and mitigate hallucinations, bias, and response inconsistencies.
Ensure quality, accuracy, and reliability of LLM outputs across business scenarios.
Support testing and validation of multi‑agent AI systems where applicable.

Skill Requirements
Skill
Mandatory
Proficiency Level (1-5)
AI/LLM Testing concepts
Must Have
4–5
LLM evaluation frameworks (DeepEval, RAGAS, TruLens, LangSmith),
Must Have
4
Prompt validation & testing
Must Have
4–5
Hallucination detection
Must Have
4
Multi-agent system validation
Valuable to Have
3–4
Other Requirements
null

📌 Senior Test Lead (Pune)
🏢 HCL Technologies
📍 Pune