Quality Assurance Engineer (India)

Quality Assurance Engineer (India)

20 Nov
|
KeeMakr AI
|
India

20 Nov

KeeMakr AI

India

QA Engineer — AI AgentsFull-time | India — Remote | Immediate joining | Minimum 2–3 years of relevant QA experienceAbout the roleWe need a QA engineer to test AI agents end to end. You check the conversation and the business app that the agent changes. You find out if the agent gives a correct, safe answer, and if the app data is correct after the chat.

About half of the work is hands-on manual testing of live apps. The other half is building and running an automated framework that runs hundreds of realistic chats and checks each result.Responsibilities• Turn test cases, PRDs and bug reports into test plans, and add the cases that the request did not name.

- Explore staging and dev apps, and map screens, roles and APIs.
- Write realistic chat scenarios. Each one starts with a real problem.
- Be sceptical of the agent: check every answer against the app data.
- Add each new check to the framework (utility, script, npm command, offline test).
- Record every chat in CSV, with the full conversation.
- Check Langfuse traces on every run. A step that is not logged is a bug.




- File clear Linear tickets that name the environment and include proof. Remove duplicates first, then retest and close or reopen each ticket.
- Respect shared environments, and keep LLM cost low.Must have• Minimum 2–3 years of relevant QA experience, covering both manual and automated testing.
- JavaScript (ESM) and Playwright.
- Experience testing LLM or chatbot products: hallucination, prompt injection, permission leaks.
- LLM-as-judge rubric design.
- API and database checks.
- Explicit bug writing and clear written English.
- Availability for immediate joining in a full-time position.Nice to have• Promptfoo and Langfuse.
- Odoo, ATS or CRM data models.
- SMS and email channel testing.
- Healthcare apps.
- AI coding agents such as Claude Code.Stack: JavaScript, Playwright, Promptfoo, Claude and OpenAI judges, Langfuse, Linear, Slack and GitHub.How to applyEmail your résumé to [HIDDEN TEXT], highlighting your relevant manual testing, automation and AI/LLM testing experience, along with your earliest joining date.

📌 Quality Assurance Engineer (India)
🏢 KeeMakr AI
📍 India

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: quality assurance engineer (india) / india

Subscribe to this job alert:

Get the latest job offers by email for: quality assurance engineer (india) / india