Role & responsibilities

Key Responsibilities: Design and implement automation frameworks for UI, API, and AI driven applications • Validate LLM responses, prompts, AI workflows, and RAG pipelines for accuracy, faithfulness, and context relevance

- Develop AI test cases covering LLM, RAG, and prompt-driven flows • Integrate LLMs with test automation frameworks to: • Auto-generate test cases • Validate AI responses • Assist in script creation using AI-assisted coding • Perform functional, regression, and exploratory testing of AI systems • Evaluate RAG performance using frameworks like RAGAS (faithfulness, context precision, answer relevance) • Create and manage test datasets, telemetry, and logs for AI validation • Collaborate with engineering, data, and ML teams to improve test reliability and AI quality Qualifications and Skills:



• Bachelors or master’s degree in computer science, Engineering, Data Science, or equivalent experience. • 6+ years of experience in QA automation or software engineering. • 3+ years of experience working with AIdriven automation or intelligent testing frameworks. • Minimum 2 years of handson experience in building or testing AIbased tools or platforms. Tools & Technologies
- - Must Have Tools •
- Selenium / Playwright / Cypress / Appium •
- REST Assured / Postman • Python / Java / JavaScript / TypeScript test frameworks •
- PyTest / JUnit / TestNG •
- Jira / TestRail •
- AWS CloudWatch / Grafana / Prometheus •
- Cursor AI / GitHub Copilot / Op

Preferred candidate profile

📌 AI Test Automation Specialist (Bengaluru)
🏢 PwC
📍 Bengaluru