About the Company

Greetings from Teamware Solutions a division of Quantum Leap Consulting Pvt. Ltd

Teamware Solutions (a division of Quantum Leap Consulting Pvt. Ltd.) is a 20+ year industry leader providing IT staffing and consulting services, supporting 4000+ professionals across India, the USA, the Middle East, and APAC for leading GCC clients.

About the Role

We are hiring an AI Quality Engineer / QE Consultant – Playwright & GenAI Testing

- Location: Bangalore
- Work Model: Hybrid
- Experience: 5 to 8 years
- Notice Period: Immediate - 15 days

Role Combination looking for QE Consultant- Playwright + API Automation + AI/GenAI Testing + LLM Evaluation + CI/CD

Responsibilities

We are seeking a highly motivated Quality Engineering (QE) Consultant with 4–6 years of experience in software testing, automation, and emerging AI technologies. The ideal candidate will have hands-on experience testing AI/GenAI applications , APIs , and web applications , along with solid expertise in Playwright automation and AI evaluation frameworks .

This role will focus on ensuring the quality, reliability, safety, and performance of AI-powered solutions by designing comprehensive test strategies, implementing automation frameworks, validating AI model behavior, and establishing evaluation mechanisms for large language model (LLM)-based applications.

The candidate will collaborate closely with Product Owners, Developers, Data Scientists, and AI Engineers to drive quality throughout the software development lifecycle.

- AI/GenAI Testing: Test LLM applications, AI agents, copilots, and conversational systems for accuracy, relevance, groundedness, hallucinations, consistency, and safety.
- Playwright Automation: Build and maintain Playwright frameworks for UI, API, and E2E automation.
- API Testing: Perform API testing using Postman, REST Assured,



and Playwright , including authentication, validation, and integration testing.
- AI Evaluation: Define AI quality metrics such as relevancy, semantic similarity, groundedness, precision/recall , and human evaluation.
- CI/CD: Integrate automation into CI/CD pipelines and support shift-left QE practices.
- Collaboration: Work with Product, Engineering, Data Science, and AI teams to identify risks and improve quality.

Experience

- 4–6 years of experience in Quality Engineering, Software Testing, or Test Automation.
- Hands-on experience testing AI/Generative AI applications.
- Experience with modern web application testing and API testing.

Technical Skills

- Strong expertise in Playwright automation.
- Hands-on experience with API testing and automation.
- Experience validating AI/LLM-based applications and AI agents.
- Good understanding of AI evaluation methodologies and testing techniques.
- Familiarity with:
- M365 Copilot
- OpenAI/Azure OpenAI
- Claude
- Gemini
- Agentic AI frameworks
- Knowledge of test management tools such as Azure DevOps (ADO), Jira, or similar platforms.
- Experience working with CI/CD pipelines and DevOps practices.

Preferred Skills

- Exposure to Python or TypeScript for automation and AI testing.
- Experience with AI evaluation frameworks such as:
- RAG evaluation
- Prompt evaluation
- LLM benchmarking
- Human feedback-based evaluation
- Knowledge of Responsible AI principles, AI governance, and model risk management.
- Experience testing Retrieval-Augmented Generation (RAG) systems and AI agents.

Education / Professional qualifications

- Bachelor's degree in Computer Science, Information Technology, Engineering, or a related discipline..

Other Information Number of interview rounds 2

Please let me know if you are interested in this position and send me your resume to [email protected]

📌 AI Quality Engineer / QE Consultant – Playwright & GenAI Testing (Bengaluru)
🏢 Teamware Solutions
📍 Bengaluru