Role: AI Quality Engineer (Tester)
Experience: 3-4 years
Location: Pune

Purpose:
Lead testing and evaluation of AI applications (LLMs, RAG, agent workflows) to ensure quality, safety, performance, and business alignment.

Key Responsibilities:

- Design & execute functional/regression test cases (SIT/UAT)
- Evaluate LLMs using frameworks (e.g., RAGAS, PromptFoo)
- Test RAG systems: retrieval accuracy, hallucination, grounding
- Validate agent workflows: multi-step logic, tool usage, structured outputs
- Perform API, performance, and load testing (latency, throughput, cost)
- Conduct security testing (prompt injection, data privacy, guardrails)
- Integrate automated tests into CI/CD pipelines
- Monitor, report, and optimize AI system performance




- Collaborate with cross-functional teams and support release readiness

Required Skills:

- 3+ years in QA/testing (API + performance testing)
- Experience with automation tools (Selenium, Playwright)
- Programming knowledge (Python/TypeScript)
- CI/CD, version control, and cloud basics (AWS/Azure/GCP)

Preferred:

- AI/ML and MLOps understanding
- Experience with LLM evaluation tools (LangSmith, RAGAS, TruLens, etc.)
- Knowledge of AI security testing and observability tools

Tools:
Playwright, Selenium, Postman, REST Assured, k6, JMeter, CI/CD tools

Traits:
Solid communication, problem-solving, and quality-focused mindset

📌 AI Quality Engineer (Mumbai)
🏢 Tekskills
📍 Mumbai