About This Role

6 to 10 years of experience
Skills: Test Automation, Selenium, Playwright, AI Testing, LLM Testing, RAG Pipeline Testing, Python, REST API Testing, RAGAS / DeepEval / LangSmith, CI/CD, BDD / Cucumber, Prompt Testing, Hallucination Detection, Shift-Left Testing, JIRA

About the Role

We are seeking a Senior Quality Engineer with 8–10 years of experience and robust AI / GenAI testing expertise to lead and elevate quality engineering practices across enterprise applications and AI-powered platforms. You will bring deep test automation skills combined with hands-on experience validating LLM outputs, RAG pipelines, GenAI applications, and ML model performance — ensuring that AI systems are accurate, reliable, fair, and safe for production use. Beyond AI testing, you will champion shift-left quality practices, design robust automation frameworks, and embed quality as a first-class concern across Agile delivery teams.



This is a high-impact role at the cutting edge of quality engineering — where traditional QE rigour meets the unique and evolving challenges of AI system validation.

Key Responsibilities

AI & GenAI Quality Assurance
- Design and implement comprehensive testing strategies for GenAI applications — covering LLM output validation, RAG pipeline testing, prompt regression, hallucination detection, and responsible AI evaluation.
- Build and maintain LLM evaluation frameworks using RAGAS, TruLens, DeepEval, or LangSmith — defining and tracking metrics for answer relevance, faithfulness, context precision, and recall across RAG and generative AI systems.
- Design prompt testing frameworks — systematically validating prompt behaviour, regression testing prompt changes, and assessing LLM output consistency and quality across model versions.
- Conduct adversarial testing and red-teaming of AI systems — probing for jailbreaks, prompt injection vulnerabilities, bias, and safety filter bypasses to ensure AI outputs meet safety and compliance standards.
-

📌 Senior Quality Engineer (Gurugram)
🏢 Lister Digital
📍 Gurugram