Job Summary

We are seeking a highly motivated and skilled Test Engineer with 3-6 years of experience to join our team, focusing on ensuring the quality, reliability, and performance of our innovative native AI-driven products within the US healthcare payer platform. This role requires a strong foundation in software testing principles, a keen eye for detail, and a proactive mindset in approaching the unique challenges of AI/ML model validation and integration testing. You will collaborate closely with product managers, data scientists, and developers to deliver high-quality, impactful AI solutions to our members and internal stakeholders.

AI Product Quality Assurance: Specializes in the comprehensive testing and validation of AI/ML-powered applications and components, ensuring they meet functional, non-functional, and ethical requirements within the sensitive US healthcare payer context.

Healthcare Payer Domain Expertise: Leverages a strong understanding of healthcare payer operations to design relevant test scenarios and validate AI solutions'' impact on claims, member services, provider networks, and regulatory compliance.

Proactive Test Strategy Design: Contributes to the development of robust test strategies, designs detailed test plans and cases, and proactively identifies potential failure points and data quality issues critical for AI/ML performance.

Automation Efficiency: Drives the adoption and implementation of test automation frameworks and scripts, particularly for data validation, API testing, and AI model inference testing, to enhance efficiency and coverage. Builds reusable AI evaluation harnesses, synthetic test datasets, regression suites for prompts/agents, CI/CD integrated AI evaluation checks.

Collaborative Communicative: Acts as a vital link between technical development teams (Data Scientists, Developers) and business stakeholders, clearly communicating test results, defects, and risks related to AI product quality. Partners with SMEs, Product, Data Science, Engineering, Architecture, AI Goverce and Operations to review AI outputs, knowledge gaps, model drift and release readiness

Responsibilities

- Test Strategy Planning: Work closely with Product Managers, Data Scientists, and Developers to understand AI product requirements, model functionalities, and system architecture. Develop comprehensive test strategies and detailed test plans for AI-driven features and end-to-end solutions.
- Test Case Design Execution: Design, develop, and execute a wide range of test cases, including functional, integration, regression, performance, and user acceptance tests,



specifically for AI/ML models and their integration into existing payer systems.
- AI/ML Model Validation Support: Contribute to the validation of AI/ML model outputs, focusing on data quality, data drift, bias detection, and performance against defined metrics.
- Design tests to evaluate model behavior under various data conditions and edge cases.
- Work with Data Scientists to understand model limitations and interpretability, translating these into test scenarios.
- AI Solution Quality Goverce Testing: Design and execute end-to-end testing of AI-driven applications, workflows, and data processes to ensure accuracy, reliability, usability, and compliance. Validate information quality, system behavior, approval workflows, auditability, and operational performance while ensuring seamless integration across business and technology components.
- Test Automation Development: Design, develop, and maintain automated test scripts and frameworks for API testing, UI testing, data validation, and automated model inference testing using relevant tools and programming languages (e.g., Python, Java, Selenium, Postman).
- Defect Management Reporting: Identify, document, and track defects with clear, concise, and reproducible steps. Collaborate with development teams to ensure timely resolution and retesting. Provide regular, clear reports on test progress, defect status, and overall product quality to project stakeholders.
- Data Validation Quality: Focus on testing data pipelines and data transformations that feed AI models, ensuring data accuracy, completeness, and consistency. (Optional but highly valued: Design and execute ETL tests).
- Performance Scalability Testing: Participate in performance and scalability testing for AI applications, ensuring they meet defined service level agreements (SLAs) under expected load.
- Continuous Improvement: Proactively identify areas for test process improvement, reject tools and techniques, and contribute to the enhancement of testing methodologies, particularly for AI product quality assurance.
- Documentation: Create and maintain clear, comprehensive test documentation, including test plans, test cases, and test reports.

Requirements
- Professional Experience (3-6 years):



3-6 years of progressive experience in software testing or quality assurance roles.
- Minimum 2-3 years directly testing applications or products within the US healthcare payer industry, demonstrating a strong understanding of claims processing, member management, provider data, and regulatory compliance (e.g., HIPAA).
- AI/ML Testing Acumen: Foundational understanding of Artificial Intelligence (AI) and Machine Learning (ML) concepts, including model types, data requirements, and the unique challenges of testing AI (e.g., non-determinism, data dependency, bias).
- Experience in testing AI/ML-powered features or products is highly preferred, even if informal.
- Ability to think proactively about how to test the ''intelligence'' and ''behavior'' of AI models, not just their functional interfaces.
- Test Design Methodologies: Proven expertise in designing comprehensive test plans and detailed test cases based on business requirements and technical specifications.
- Strong understanding of various testing methodologies (Agile, Waterfall) and different types of testing (functional, non-functional, integration, regression, UAT).
- Strong understanding of AI testing layers: data/knowledge layer testing, model testing, agent testing, application/workflow testing, end-to-end AI system testing, and responsible AI/goverce testing.
- Automation Experience: Hands-on experience in developing and maintaining automated test scripts using programming languages like Python or Java.
- Familiarity with test automation frameworks and tools (e.g., Selenium, Playwright, Cypress, Postman for API testing).
- Technical Skills: Solid proficiency in SQL for data validation, querying databases, and analyzing test data. (Minimum 2 years' experience).
- Experience with API testing tools (e.g., Postman, SoapUI).
- Familiarity with version control systems (e.g., Git).
- Foundational understanding of GCP/Vertex AI, embeddings/vector stores, knowledge graphs/Neo4j or equivalent graph solutions,
- Communication Collaboration: Excellent verbal and written communication skills, with the ability to clearly articulate technical issues, test results, and risks to diverse audiences, including technical and non-technical stakeholders.
- Robust interpersonal skills and proven ability to collaborate effectively within cross-functional teams (Product, Data Science, Engineering).
- Problem-Solving Analytical Skills: Exceptional analytical skills to diagnose complex issues, identify root causes, and propose effective testing solutions.
- A meticulous and detail-oriented approach to quality assurance.

📌 Consultant - Test Engineer_Nexus AI (Chennai)
🏢 enGen Global
📍 Chennai