Role Overview We are looking for an experienced
QA Manager
to lead our quality engineering function, with a strong focus on
test automation, evaluation of AI/LLM-based systems, and team leadership . The ideal candidate combines deep hands-on technical expertise with proven experience managing and mentoring QA teams. Key Details Position:
QA Manager – Automation Total Experience:
10 to 17 years Team Management Experience:
Minimum 2 years (leading QA/automation teams) Employment Type:
Full-time Roles & Responsibilities Lead, mentor, and grow a team of QA/automation engineers, driving their technical and career development. Define and own the overall test automation strategy, framework architecture, and quality roadmap for the product. Design, build, and maintain scalable automation frameworks using
Playwright
for web application testing. Drive
evaluation (eval) strategies
for AI/LLM-based features — designing test harnesses, golden datasets, scoring rubrics, and regression suites for model outputs. Partner with data science/ML engineering teams to define quality benchmarks for
LLM-powered and AI-driven systems , including hallucination checks, output consistency, and prompt regression testing. Write and review robust, maintainable automation code in
Java , integrating with CI/CD pipelines. Establish QA processes, best practices, and metrics (coverage, defect leakage, automation ROI) across the team. Collaborate closely with Product, Engineering, and Data Science stakeholders to define acceptance criteria and quality gates, including for AI-driven features. Own test planning, risk assessment, release sign-off,



and reporting to leadership. Stay current with emerging tools and techniques in AI/LLM testing and evaluation methodologies. Required Skills & Experience 10–17 years
of total QA/testing experience, including automation-focused roles. Minimum 2 years
of experience managing/leading a QA or automation team. Strong hands-on experience with
Playwright
for automated UI/end-to-end testing. Solid
Java
programming skills for building and maintaining test frameworks. Demonstrated experience
testing AI/LLM-based systems
— prompt testing, model output validation, regression testing for generative AI features. Practical knowledge of
eval skills
— designing evaluation frameworks, benchmarks, scoring criteria, and golden datasets for AI systems. Working knowledge of
LLM concepts
(prompting, context windows, hallucinations, RAG, fine-tuning basics) sufficient to design meaningful test/eval strategies. Experience with CI/CD integration for automated test suites (Jenkins, Git Hub Actions, or similar). Strong understanding of QA methodologies: functional, regression, integration, performance, and exploratory testing. Excellent communication and stakeholder management skills. Good to Have Experience with additional automation tools (Selenium, Cypress, etc.). Exposure to API testing tools (Postman, REST Assured). Familiarity with cloud platforms (AWS/Azure/GCP) and containerized test environments. Prior experience in a product-based or SaaS company testing AI-powered features. Qualifications Bachelor's/Master's degree in Computer Science, Engineering, or a related field (or equivalent practical experience).

📌 Quality Assurance Manager (Bengaluru)
🏢 Tekion
📍 Bengaluru