Overview

Quality Assurance Engineer owns quality end-to-end — from test strategy through release sign-off — for AI-driven insurance workflows. They are responsible for validating agentic / LLM-based systems where outputs are probabilistic, workflows are autonomous, and accuracy is business-critical. They also need too make sure that flows are meeting the expectations of the business as well as IT owners of the customer organizations

Key Responsibilities

- Define and own the overall test strategy for the platform, covering functional, regression, integration, API, UAT and AI/agent behavior testing

- Create detailed test plans, test designs, test scenarios and test cases mapped to business requirements and workflows for insurance workflows (submissions, quoting, policy servicing, endorsements, renewals, claims/FNOL, COIs)

- Design evaluation frameworks for agentic AI outputs — accuracy, hallucination detection, grounding, guardrail adherence, prompt/response regression and human-in-the-loop checkpoints

- Validate integrations with AMS platforms (Applied Epic, AMS360, HawkSoft, etc.), carrier portals, ACORD forms and document extraction pipelines

- Manage defect lifecycle: triage, root-cause analysis, severity/priority assessment and closure tracking with engineering and product teams

- Define QA metrics and reporting (test coverage, defect density, escape rate, AI eval scores) and drive release readiness / go-no-go decisions

- Support client UAT during implementations, including test data preparation with realistic datasets for insurance domain/edge-case coverage





- Build and maintain automation suites (UI, API, and data validation) and integrate them into CI/CD pipelines

- Mentor junior QA engineers and establish QA best practices, standards and documentation.

Required Skills & Experience

Must Have

- 3–5 years of QA/testing experience

- Demonstrated experience creating test strategy, test plans and test design from scratch for complex enterprise products

- Hands-on experience testing agentic AI / LLM-based platforms — evaluating non-deterministic outputs, prompt regression, RAG/grounding accuracy and agent workflow correctness

- Robust API testing skills (Postman/REST Assured), SQL for data validation

- Experience with test management and defect tools (Jira, TestRail/Zephyr/Xray) and Agile/Scrum delivery

- Excellent analytical, documentation and communication skills; able to work directly with product, engineering and client teams

Good to Have

- 2+ years’ experience in testing applications in the U.S. P&C; insurance domain (broker, carrier, or MGA systems)

- Automation experience using tools such as Selenium/Playwright/Cypress or similar

- Familiarity with insurance data standards and artifacts: ACORD forms, policy documents, loss runs and carrier downloads.

- Exposure to AI evaluation tooling/frameworks (e.g., LLM eval harnesses, golden datasets, model output scoring)

- Experience in client-facing implementation or UAT support roles; ISTQB or equivalent certification.

Pay: From ₹8,000,000.00 per year

Work Location: In person

📌 QA Engineer (Pune)
🏢 Ideas to Impacts
📍 Pune