Role: Sr. AI QA Engineer
Location: Noida / Gurugram / Pune / Mumbai
Employment Type: Full-Time

About the Role
We are looking for a highly skilled AI QA Engineer to join our Quality Engineering team. This role combines traditional automation testing expertise with modern AI application validation. The ideal candidate will have hands-on experience testing GenAI applications, LLM-powered systems, chatbots, RAG applications, and AI-driven workflows , along with strong automation experience using Playwright .

You will work closely with Product, Engineering, Data Science, and AI teams to ensure the quality, reliability, and accuracy of next-generation AI-powered products.

Key Responsibilities
Design, develop, and maintain test automation frameworks using Playwright and Selenium.
Perform end-to-end testing of web applications and AI-powered user experiences.
Validate chatbot conversations, AI-generated responses, and conversational workflows.
Execute testing for GenAI, LLM, and RAG (Retrieval-Augmented Generation) applications.
Perform prompt testing, grounding validation, and hallucination detection.
Validate AI agent workflows, multi-step reasoning, and tool integrations.
Execute functional, regression, integration, API, database, and UAT testing.
Perform SQL-based backend validations and data quality checks.
Collaborate with cross-functional teams to define and implement quality strategies.
Participate in Agile ceremonies and contribute to continuous quality improvement.

Must-Have Qualifications
Bachelor's degree in Engineering ( B.E./B.Tech ) in Computer Science, Information Technology, Electronics, or related fields.
Strong hands-on experience with Playwright (Mandatory).
Experience with Selenium Automation .




Experience testing GenAI, LLM, Chatbot, or AI-powered applications .
Hands-on experience with RAG Testing .
Strong understanding of:
Prompt Testing
Hallucination Testing
Response Validation
Grounding Validation
Experience with API Testing using Postman or REST APIs.
Strong SQL skills and experience with databases such as:
MySQL
PostgreSQL
MongoDB
DynamoDB
Experience with Python or Java .
Familiarity with:
Git
Jira
CI/CD Pipelines
Agile Methodologies

Good-to-Have
Experience with Agentic AI Testing
Experience with MCP (Model Context Protocol)
Exposure to LangSmith, Ragas, Promptfoo, DeepEval
Experience with vector databases:
Pinecone
ChromaDB
Weaviate
FAISS
AWS or GCP experience
Exposure to Amazon Bedrock or Azure OpenAI

Preferred Project Experience
Candidates with experience in any of the following will be highly preferred:
GenAI & Chatbot Testing
Conversational AI testing
Prompt validation
LLM response evaluation
Hallucination detection
RAG Application Testing
Retrieval quality validation
Embedding and chunking validation
Knowledge grounding verification
Citation accuracy testing
Agentic AI Testing
Multi-agent workflows
Autonomous decision-making systems
AI workflow orchestration
Tool and API integrations

Skills
Playwright, Selenium, Generative AI, LLM Testing, RAG Testing, AI Testing, Chatbot Testing, Prompt Testing, Agentic AI, Python, Java, API Testing, Postman, SQL, AWS, CI/CD, Jira, Test Automation, Quality Assurance

Why Join Us?
Join a team building creative AI-powered products where you'll play a critical role in ensuring reliability, accuracy, and exceptional user experiences through advanced AI quality engineering practices.

📌 Senior AI QA Engineer (Kanpur)
🏢 Biz2X
📍 Kanpur