The client is looking at hiring a AI Quality & Model Evaluation Lead for their team at Bangalore.

Experience : 5+ Years

Mandatory Skills:

- QA Strategy, Functional/API/Regression Testing
- AI/ML & LLM Evaluation
- Healthcare RCM & Audit Processes
- SQL & Data Analysis
- Statistical Sampling & Quality Assessment
- Test Management & Defect Tracking
- Leadership & Stakeholder Management

Role Summary : -QA & Software Testing (Functional, integration, regression, API, data validation, test planning and execution)

-AI/ML Model Evaluation (Model accuracy, precision, recall, F1-score, sensitivity, specificity, benchmark and ground-truth validation)

-LLM & Generative AI Evaluation (Response accuracy, relevance, groundedness, hallucination, factuality, consistency, and prompt evaluation)

-Healthcare RCM (Medical coding validation, clinical documentation review, AR denial management, Prior Authorization and audit methodology etc)

-Statistical Sampling & Audit (Random/stratified sampling, sample size determination, quality audits, confidence levels, and error analysis)

-Data Analysis & SQL (Data extraction, validation, reconciliation, quality analysis, and complex SQL queries)

-Python (Data analysis, statistical analysis, test automation, and evaluation scripts)

-LLM Evaluation Tools (DeepEval, Ragas, LangSmith, or equivalent evaluation frameworks)

-API Testing (Postman, REST APIs, request/response validation, authentication, and API automation)

-Test Management & Defect Tracking (JIRA, TestRail, TestLink, defect lifecycle,



test execution, and reporting)

-QA Automation (Automation frameworks, regression automation, data validation, and test execution optimization)

-Quality Metrics & Reporting (Test coverage, defect trends, model accuracy, audit results, quality KPIs, and dashboards)

-Go/No-Go & Risk Assessment (Release readiness, quality risk evaluation, acceptance criteria, and data-driven decisions)

-Agile & QA Process (Sprint planning, estimation, QA processes, continuous improvement, and cross-functional collaboration)

-Leadership & Stakeholder Management (Team mentoring, task planning, communication, issue escalation, and stakeholder coordination)

Mandatory Skills:

-QA & Test Strategy (Test planning, test case design, functional/API/regression testing, defect management, and QA processes) — Mandatory

-AI/ML & LLM Evaluation (Model validation, accuracy metrics, ground-truth comparison, LLM response evaluation, hallucination, relevance, and groundedness) — Mandatory

-Healthcare RCM & Audit (Medical coding validation, clinical documentation review, AR denial management, Prior Authorization and audit methodology etc) — Mandatory

-SQL & Data Analysis (Data validation, EDA, cleaning, aggregation, reconciliation, quality analysis, and model-result analysis) — Mandatory

-Statistical Sampling & Quality Decision-Making (Sampling methodology, statistical analysis, quality thresholds, risk assessment, and Go/No-Go decisions) — Mandatory

📌 AI Quality & Model Evaluation Lead (US IT Healthcare) (Bangalore Metropolitan Area)
🏢 iSoft Globl
📍 Bangalore Metropolitan Area