AI QA Senior

Educational qualification: BTech/Masters/PhD

The opportunity

We are seeking an experienced AI QA Engineer with 5–9+ years of professional experience and deep expertise in AI testing, quality engineering, validation frameworks, and enterprise AI solution assurance. The ideal candidate should possess strong hands-on experience validating AI systems, Agentic AI workflows, RAG applications, conversational AI solutions, and enterprise-scale AI platforms.

The individual will play a key role in ensuring the quality, reliability, security, safety, and performance of AI systems while collaborating with AI engineers, architects, developers, and business stakeholders to establish robust quality assurance practices across AI and Generative AI implementations.

Important note: This role requires skilled implementation experience in relevant AI technologies. Personal portfolio projects, cloned demos, certifications, or tutorial-based builds are helpful, but are not sufficient on their own.

Application guidance:



Candidates should be able to clearly explain at least two relevant AI implementations, including the business use case, their direct contribution, the technical approach, and the outcome delivered.

Your key responsibilities
- Design, develop, and execute comprehensive testing strategies for AI, GenAI, Agentic AI, and Agentic RAG systems.
- Validate Large Language Model outputs for accuracy, consistency, relevance, safety, and compliance requirements.
- Develop test frameworks and automation suites for AI services, copilots, AI assistants, and enterprise AI applications.
- Design and execute evaluation frameworks for RAG systems, retrieval pipelines, vector search solutions, and knowledge-based AI applications.
- Perform functional, integration, regression, performance, reliability, and scalability testing of AI-powered applications.
- Develop automated testing frameworks using Python to validate AI workflows, APIs, models, and orchestration pipeline

📌 AI QA Engineer (Mumbai)
🏢 EY
📍 Mumbai