Role Overview: We are seeking an experienced AI Test Lead to lead end-to-end testing and quality assurance of Generative AI applications. This role bridges AI quality engineering, test automation, and customer engagement. You will collaborate with QA Managers, AI/ML Engineers, Prompt Engineers, Full Stack Developers, and DevOps Engineers to validate production-ready, scalable AI applications against industry AI testing standards, while serving as the primary testing point of contact for customers.

Key Responsibilities

Define and execute AI testing strategy, planning, execution, defect management, and release sign-off.

Lead functional, automation, API, integration, performance, and AI-specific testing for GenAI applications.

Design test approaches for LLM evaluation, prompt regression, hallucination detection, bias and fairness testing, robustness validation, and output quality assessment.

Manage sprint-level test planning, estimation, quality metrics, and release readiness in Agile environments.

Act as the primary QA contact for customers, driving requirement discussions, status reviews, defect triage, and UAT support.

Build and enhance Python-based automation frameworks using Pytest, Playwright, Selenium, and API automation tools.

Implement automated AI evaluation pipelines using tools such as DeepEval, Ragas, LangSmith, or promptfoo.





Integrate test automation into CI/CD pipelines and establish quality governance, documentation, and reporting standards.

Mentor QA engineers and promote best practices in AI testing and automation.

Required skills

8–15+ years in software testing and QA, with experience leading testing teams.

Hands-on experience in AI/GenAI testing, LLM validation, or intelligent application testing.

Strong Python programming and test automation expertise.

Experience with API testing, automation frameworks, and CI/CD pipelines.

Solid customer communication, stakeholder management, and delivery leadership skills.

Experience with Agile/Scrum methodologies and test management tools.

Preferred qualifications

Experience with RAG systems, conversational AI, AI agents, or LLM-based applications.

Familiarity with AI governance frameworks such as ISO/IEC 42001, NIST AI RMF, or EU AI Act.

Knowledge of AI quality metrics including accuracy, relevance, groundedness, consistency, and toxicity evaluation.

What This Role

Offers

Lead cutting-edge GenAI testing programs with high visibility to customers and senior leadership

Ownership of AI quality strategy from test planning through production sign-off

Work with talented AI architects, engineers, and domain experts

Continuous learning and growth in the rapidly evolving AI testing and quality engineering space

📌 GenAI Test Lead (Hyderabad)
🏢 QualiZeal
📍 Hyderabad