Role: Sr. AI QA Engineer

Location: Noida / Gurugram / Pune / Mumbai

Employment Type: Full-Time

About the Role

We are looking for a highly skilled AI QA Engineer to join our Quality Engineering team. This role combines traditional automation testing expertise with modern AI application validation. The ideal candidate will have hands-on experience testing GenAI applications, LLM-powered systems, chatbots, RAG applications, and AI-driven workflows, along with strong automation experience using Playwright.

You will work closely with Product, Engineering, Data Science, and AI teams to ensure the quality, reliability, and accuracy of next-generation AI-powered products.

Key Responsibilities

- Design, develop, and maintain test automation frameworks using Playwright and Selenium. - Perform end-to-end testing of web applications and AI-powered user experiences. - Validate chatbot conversations, AI-generated responses, and conversational workflows. - Execute testing for GenAI, LLM, and RAG (Retrieval-Augmented Generation) applications. - Perform prompt testing, grounding validation, and hallucination detection. - Validate AI agent workflows, multi-step reasoning, and tool integrations. - Execute functional, regression, integration, API, database, and UAT testing. - Perform SQL-based backend validations and data quality checks. - Collaborate with cross-functional teams to define and implement quality strategies. - Participate in Agile ceremonies and contribute to continuous quality improvement.

Must-Have Qualifications

- Bachelor's degree in Engineering (B.E./B.Tech) in Computer Science, Information Technology, Electronics, or related fields. - Strong hands-on experience with Playwright (Mandatory). - Experience with Selenium Automation.



- Experience testing GenAI, LLM, Chatbot, or AI-powered applications. - Hands-on experience with RAG Testing. - Strong understanding of: - Prompt Testing - Hallucination Testing - Response Validation - Grounding Validation - Experience with API Testing using Postman or REST APIs. - Solid SQL skills and experience with databases such as: - MySQL - PostgreSQL - MongoDB - DynamoDB - Experience with Python or Java. - Familiarity with: - Git - Jira - CI/CD Pipelines - Agile Methodologies

Good-to-Have

- Experience with Agentic AI Testing - Experience with MCP (Model Context Protocol) - Exposure to LangSmith, Ragas, Promptfoo, DeepEval - Experience with vector databases: - Pinecone - ChromaDB - Weaviate - FAISS - AWS or GCP experience - Exposure to Amazon Bedrock or Azure OpenAI

Preferred Project Experience

Candidates with experience in any of the following will be highly preferred:

GenAI & Chatbot Testing

- Conversational AI testing - Prompt validation - LLM response evaluation - Hallucination detection

RAG Application Testing

- Retrieval quality validation - Embedding and chunking validation - Knowledge grounding verification - Citation accuracy testing

Agentic AI Testing

- Multi-agent workflows - Autonomous decision-making systems - AI workflow orchestration - Tool and API integrations

Skills

Playwright, Selenium, Generative AI, LLM Testing, RAG Testing, AI Testing, Chatbot Testing, Prompt Testing, Agentic AI, Python, Java, API Testing, Postman, SQL, AWS, CI/CD, Jira, Test Automation, Quality Assurance

Why Join Us?

Join a team building innovative AI-powered products where you'll play a critical role in ensuring reliability, accuracy, and exceptional user experiences through advanced AI quality engineering practices.

📌 Senior AI QA Engineer (Noida)
🏢 Biz2X
📍 Noida