Role & responsibilities

Key Responsibilities: Design and implement automation frameworks for UI, API, and AI driven applications • Validate LLM responses, prompts, AI workflows, and RAG pipelines for accuracy, faithfulness, and context relevance
Develop AI test cases covering LLM, RAG, and prompt-driven flows • Integrate LLMs with test automation frameworks to: • Auto-generate test cases • Validate AI responses • Assist in script creation using AI-assisted coding • Perform functional, regression, and exploratory testing of AI systems • Evaluate RAG performance using frameworks like RAGAS (faithfulness, context precision, answer relevance) • Create and manage test datasets, telemetry, and logs for AI validation • Collaborate with engineering, data, and ML teams to improve test reliability and AI quality Qualifications and Skills:



• Bachelors or master’s degree in computer science, Engineering, Data Science, or equivalent experience. • 6+ years of experience in QA automation or software engineering. • 3+ years of experience working with AIdriven automation or intelligent testing frameworks. • Minimum 2 years of handson experience in building or testing AIbased tools or platforms. Tools & Technologies
- Must Have Tools •
Selenium / Playwright / Cypress / Appium •
REST Assured / Postman • Python / Java / JavaScript / TypeScript test frameworks •
PyTest / JUnit / TestNG •
Jira / TestRail •
AWS CloudWatch / Grafana / Prometheus •
Cursor AI / GitHub Copilot / Op

Preferred candidate profile

📌 Ai Test Automation Specialist Bengaluru
🏢 PwC
📍 Bengaluru