Job Description

Role: Sr. AI QA Engineer Location: Noida / Gurugram / Pune / Mumbai Employment Type: Full-Time About the Role We are looking for a highly skilled AI QA Engineer to join our Quality Engineering team. This role combines traditional automation testing expertise with modern AI application validation. The ideal candidate will have hands-on experience testing GenAI applications, LLM-powered systems, chatbots, RAG applications, and AI-driven workflows, along with strong automation experience using Playwright. You will work closely with Product, Engineering, Data Science, and AI teams to ensure the quality, reliability, and accuracy of next-generation AI-powered products. Key Responsibilities - Design, develop, and maintain test automation frameworks using Playwright and Selenium. - Perform end-to-end testing of web applications and AI-powered user experiences. - Validate chatbot conversations, AI-generated responses, and conversational workflows. - Execute testing for GenAI, LLM, and RAG (Retrieval-Augmented Generation) applications. - Perform prompt testing, grounding validation, and hallucination detection. - Validate AI agent workflows, multi-step reasoning, and tool integrations. - Execute functional, regression, integration, API, database, and UAT testing. - Perform SQL-based backend validations and data quality checks. - Collaborate with cross-functional teams to define and implement quality strategies. - Participate in Agile ceremonies and contribute to continuous quality improvement. Must-Have Qualifications - Bachelor's degree in Engineering (B.E./B.Tech) in Computer Science, Information Technology, Electronics, or related fields. - Robust hands-on experience with Playwright (Mandatory). - Experience with Selenium Automation.



- Experience testing GenAI, LLM, Chatbot, or AI-powered applications. - Hands-on experience with RAG Testing. - Strong understanding of: - Prompt Testing - Hallucination Testing - Response Validation - Grounding Validation - Experience with API Testing using Postman or REST APIs. - Strong SQL skills and experience with databases such as: - MySQL - PostgreSQL - MongoDB - DynamoDB - Experience with Python or Java. - Familiarity with: - Git - Jira - CI/CD Pipelines - Agile Methodologies Good-to-Have - Experience with Agentic AI Testing - Experience with MCP (Model Context Protocol) - Exposure to LangSmith, Ragas, Promptfoo, DeepEval - Experience with vector databases: - Pinecone - ChromaDB - Weaviate - FAISS - AWS or GCP experience - Exposure to Amazon Bedrock or Azure OpenAI Preferred Project Experience Candidates with experience in any of the following will be highly preferred: GenAI & Chatbot Testing - Conversational AI testing - Prompt validation - LLM response evaluation - Hallucination detection RAG Application Testing - Retrieval quality validation - Embedding and chunking validation - Knowledge grounding verification - Citation accuracy testing Agentic AI Testing - Multi-agent workflows - Autonomous decision-making systems - AI workflow orchestration - Tool and API integrations Skills Playwright, Selenium, Generative AI, LLM Testing, RAG Testing, AI Testing, Chatbot Testing, Prompt Testing, Agentic AI, Python, Java, API Testing, Postman, SQL, AWS, CI/CD, Jira, Test Automation, Quality Assurance Why Join Us? Join a team building innovative AI-powered products where you'll play a critical role in ensuring reliability, accuracy, and exceptional user experiences through advanced AI quality engineering practices.

📌 Senior AI QA Engineer (Noida)
🏢 Biz2X
📍 Noida