Must have 3+ years in software QA/testing, with hands-on senior-level experience evaluating LLM-based or agentic AI systems (not just traditional QA)
Direct experience with eval frameworks/tools LangSmith strongly preferred; also acceptable:
- Ragas, DeepEval, PromptFoo, Braintrust, Langfuse
- Solid understanding of RAG architecture and how to independently evaluate retrieval quality vs. generation quality
- Practical familiarity with MCP (Model Context Protocol) or comparable agent tool calling frameworks
- Strong Python skills for building test harnesses, scoring pipelines, and automation
- Experience designing evaluation approaches for non-deterministic systems (probabilistic scoring, not binary pass/fail)
- Comfortable working directly with client engineering teamsstrong communication, able to explain quality risk to both technical and business stakeholders;
📌 AI Test Engineer (Chennai)
🏢 Golden Opportunities
📍 Chennai
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.