– QA Manager (Automation + LLM testing)

Position: QA Manager

We are looking for a highly driven and technically strong QA Manager to lead quality engineering initiatives across modern web applications and AI-powered products. The ideal candidate should have robust expertise in test automation using Playwright with JavaScript, hands-on understanding of GenAI/LLM evaluation methodologies, and excellent stakeholder communication skills. This role requires someone who can thrive in a fast-paced, high-growth environment while building scalable quality processes and collaborating across engineering, product, and client teams.

Key Responsibilitie

- sLead and manage end-to-end Quality Assurance activities across multiple products and releases
- .Define and implement QA strategies, test plans, automation frameworks, and quality standards
- .Drive automation initiatives using Playwright and JavaScript for UI, API, and end-to-end testing
- .Establish scalable automation practices integrated with CI/CD pipelines
- .Validate and evaluate GenAI/LLM-based applications using metrics such as relevancy, faithfulness, contextual accuracy, hallucination detection, bias, and response quality
- .Collaborate with Product Managers, Engineering teams, Data/AI teams, and clients to ensure high-quality product delivery
- .Participate in requirement discussions and provide quality-focused inputs during early development stages
- .Manage defect lifecycle, root cause analysis, risk assessment, and release sign-offs
- .Build and mentor high-performing QA teams while promoting a strong quality culture
- .Work closely with clients and stakeholders to provide testing updates, quality reports, risk analysis, and release readiness communication
- .Adapt quickly to evolving priorities and support rapid product iterations in a fast-growing environment




- .Drive continuous improvement initiatives across testing processes, tools, and delivery practices
- .Ensure application performance, reliability, security, usability, and compliance standards are met
- .Track and report QA metrics, automation coverage, release quality, and operational efficiency

.

Required Skills & Qualificatio

- nsBachelor’s degree in Computer Science, Engineering, or related fiel
- d.10+ years of experience in Software Testing/Quality Engineering with leadership experienc
- e.Strong hands-on experience with Playwright and JavaScript/TypeScript automation framework
- s.Good understanding of API testing, UI testing, regression testing, and CI/CD integratio
- n.Practical understanding of Large Language Models (LLMs), GenAI workflows, prompt validation, and AI evaluation concept
- s.Experience with LLM evaluation tools/frameworks such as DeepEval or equivalent is preferre
- d.Excellent client-facing communication and stakeholder management skill
- s.Strong collaboration and cross-functional coordination abilitie
- s.Ability to work effectively in fast-paced startup or high-growth environment
- s.Experience in Agile/Scrum development methodologie
- s.Strong analytical, problem-solving, and decision-making capabilitie

s. Preferred Qualificati

- onsExperience testing AI-powered healthcare, SaaS, or enterprise applicatio
- ns.Exposure to cloud platforms such as AWS, Azure, or G
- CP.Knowledge of performance testing and security testing practic
- es.Familiarity with accessibility testing and modern quality engineering practic
- es.Experience building QA processes from scratch in scaling organizatio

ns.

What We’re Looking

- ForOwnership mindset with strong execution capabilit
- ies.Passion for innovation and modern quality engineering practi
- ces.Ability to balance technical depth with people leaders
- hip.Strong focus on delivering exceptional product quality and customer experie

nce.

📌 QA Automation Manager (LLM Testing ) (India)
🏢 Ekshvaku Tech Innovations
📍 India