Responsibilities We are seeking a highly motivated Quality Engineering QE Consultant with 46 years of experience in software testing automation and emerging AI technologies The ideal candidate will have handson experience testing AIGenAI applications APIs and web applications along with strong expertise in Playwright automation and AI evaluation frameworks
This role will focus on ensuring the quality reliability safety and performance of AIpowered solutions by designing comprehensive test strategies implementing automation frameworks validating AI model behavior and establishing evaluation mechanisms for large language model LLMbased applications The candidate will collaborate closely with Product Owners Developers Data Scientists and AI Engineers to drive quality throughout the software development lifecycle
AI GenAI Testing
Design and execute test strategies for AIGenAI applications including LLMpowered solutions AI agents copilots and conversational systems
Validate AI outputs for
Accuracy
Relevance
Groundedness
Consistency
Hallucination detection
Toxicity and safety compliance
Create and maintain prompt test suites and benchmark datasets
Perform functional regression performance and reliability testing for AIenabled applications
Define and execute AI evaluation metrics and quality gates
Automation Quality Engineering
Develop and maintain test automation frameworks using Playwright
Automate UI API and endtoend business workflows
Create reusable automation assets and testing accelerators
Integrate automated tests into CICD pipelines
Support shiftleft testing and quality engineering practices
API Testing
Design and execute API test cases using tools such as
- Postman
- REST Assured
- Playwright API Testing
Validate API contracts authentication authorization error handling and performance Conduct integration testing across distributed systems and thirdparty services
AI Evaluation Quality Metrics
Establish AI evaluation frameworks and quality scorecards
Measure model quality using evaluation techniques such as