HIRING ALERT AI QA ENGINEER

Role: AI QA Engineer
Work Location: Navi Mumbai/ Bangalore
Experience: 3-6 Years

Preferred Immediate Joiner.

Applicants please share your updated resumes on [email protected]

Role Overview
We are looking for an AI QA Engineer with hands-on experience working on GenAI platforms and workflows, particularly AI observability, evaluation, testing, or safety platforms similar in nature to Arize, Cekura, Fiddler, Giskard, WhyLabs, Phoenix, or equivalent systems.
This role requires direct exposure to GenAI bots, RAG pipelines, and AI workflows in real platforms, not theoretical knowledge or generic QA experience. You will evaluate AI behavior, analyze failures, and contribute to improving the quality, reliability, and safety of GenAI systems.
This is not a traditional QA role and not suitable for candidates who have only tested UI, APIs, or deterministic systems.

Key Responsibilities
Execute hands-on AI evaluation and QA on:
GenAI bots (chatbots, copilots, agents)
RAG-based workflows
AI evaluation and observability platforms
Design and execute:




Prompt-based test scenarios
Multi-turn conversation tests
Edge-case and adversarial evaluations
Evaluate AI behavior for:
Hallucinations
Grounding failures
Inconsistency and drift
Safety and reliability issues
Work within existing AI platforms (evaluation, observability, monitoring) to:
Analyze traces, metrics, and outputs
Interpret model behavior across runs
Document findings clearly in:
Evaluation reports
Platform dashboards
Structured test summaries
Collaborate with:
AI QA Lead on methodology and review
ML and Product teams to provide actionable feedback

Must-Have Skills & Experience
3–5+ years of experience in AI QA, AI evaluation, or GenAI testing
Hands-on experience working on GenAI platforms, such as:
AI evaluation platforms
AI observability or monitoring tools
GenAI safety or reliability platforms
Direct experience testing:
GenAI chatbots or agents
RAG pipelines and workflows
M

📌 AI QA Engineer (Navi Mumbai)
🏢 QualityKiosk Technologies
📍 Navi Mumbai