Job Summary

We are looking for a QA Automation Engineer to own fully automated testing of our AI products. This is an automation-only role with no manual testing. All functional, API, regression and LLM output tests must run automatically in the CI/CD pipeline and act as quality gates before every release.

Key Responsibilities

- Design and own the test automation strategy and framework for AI products.
- Build automated UI, API and end-to-end test suites using Playwright/Selenium and PyTest.
- Automate LLM and GenAI output evaluation, covering accuracy, relevance, hallucination, safety and prompt regression.
- Integrate all test suites into CI/CD pipelines (Azure DevOps/GitHub Actions) as release quality gates.
- Build automated regression, smoke and performance tests, including test data management.
- Track and report test coverage, pass rates and defect trends through dashboards.
- Work with AI/ML and DevOps engineers on shift-left testing and fast feedback loops.

Required Skills





- Solid test automation with Playwright and/or Selenium.
- Python with PyTest; API test automation (REST Assured, Postman/Newman or Python requests).
- CI/CD integration of tests in Azure DevOps or GitHub Actions.
- LLM evaluation testing using tools such as DeepEval, Ragas, promptfoo or similar.
- Git, test reporting (Allure or similar) and Agile/Scrum practices.
- Clear understanding of test design, coverage and defect lifecycle.

Preferred Skills

- Performance testing (JMeter, Locust or k6).
- Security test basics (OWASP ZAP) and accessibility testing.
- Testing in Docker/Kubernetes environments.
- ISTQB Advanced Test Automation Engineer certification.

Educational Qualification

B.E. / B.Tech. / M.E. / M.Tech. / MCA , IT, AI, Data Science or a related discipline. ISTQB or an equivalent test automation certification is a plus.

📌 Qa Test Engineer (Gurugram)
🏢 PwC India
📍 Gurugram