Job Summary

We are looking for an experienced AI Automation Tester with hands-on experience in testing AI-powered products and applications. The ideal candidate should have solid expertise in Playwright automation, along with practical experience in LLM evaluations (Evals), Golden Datasets (Goldens), and AI Observability.

Key Responsibilities
Design, develop and execute automated test scripts using Playwright.
Test AI-powered applications, LLM-based products and Generative AI solutions.
Work on Evals to assess AI response accuracy, relevance, quality and consistency.
Create, maintain and validate Golden Datasets (Goldens) for benchmarking AI outputs.
Implement and work with AI Observability to monitor model performance, response quality, latency and failures.
Perform functional, regression, API and end-to-end testing.
Identify, report and track defects to ensure product quality and reliability.

Skills:- Playwright, evals, golden, Large Language Models (LLM) and AI product testing

📌 Automation And Ai Product Testing Bengaluru
🏢 Nasugroupcom
📍 Bengaluru