Job Requirements

Senior/Lead QA Automation Engineer AI/ML & GenAI

Power Systems AI HUB

Power Systems builds highly available, massively scalable AI digital platforms for the enterprise. As part of our AI Hub organization, we are looking for a highly skilled Automation Test Engineer to support our AI/ML and GenAI product ecosystem.

Role Overview

We are seeking a Senior QA Engineer – Automation Testing with 5–6 years of experience in designing, developing, and executing automation frameworks for AI/ML-driven applications. The ideal candidate brings hands-on experience with GenAI Testing, LangSmith, PyTest, Agentic frameworks, and strong AI/ML evaluation expertise.

Key Responsibilities

AI / GenAI Testing

- Design and execute V&V; model strategies for classical ML and GenAI/ agentic systems.
- Define test plans, tests cases, and execute them
- Evaluate AI models for accuracy, performance, safety, robustness, hallucinations, bias, and correctness.
- For GenAI, rely on LangSmith for LLM observability, tracing, monitoring, and evaluation, for offline and online evaluation
- For classical ML UC, ensure V&V; tests are done during the whole life cycle of the model (ML monitoring)

Test Automation

- Build scalable automation frameworks using Python & PyTest integrated with CI/CD pipelines.
- Develop automated test suites for APIs, microservices, and AI pipelines.
- Perform sanity, integration, functional, regression, and UAT testing across QA, UAT, and Prod environments.
- Implement automation strategies: Keyword-driven,



Data-driven, BDD, Hybrid frameworks.

Quality Assurance & Reporting
- Analyze test results and provide insights to engineering teams.
- Log and track defects using standard bug-reporting tools.
- Maintain high-quality documentation for all test processes and reports.

Required Skills & Experience
- 5–6 years of experience in software testing & automation.
- Strong programming expertise in Python.
- Valuable understanding of Data Science related metrics (for both classical ML and GenAI UC) Experience with AI/Chatbot products and testing AI applications.
- Experience with Git, GitHub Copilot, and CI/CD pipelines.
- Knowledge of Azure/AWS cloud environments.
- Familiarity with MySQL, PostgreSQL, Oracle, SQL Server, NoSQL databases.
- Solid verbal & written communication skills.
- Experience with multi-stakeholder delivery models including onshore–offshore.

Preferred

- Experience in AI/ML model evaluation.
- Exposure to AI observability tools and automated prompt validation.
- Hands-on expertise with LangSmith, Agentic frameworks, PyTest, BDD/V&V; frameworks.

Soft Skills

- Analytical & Critical Thinking
- Curiosity & Continuous Learning
- Strategic Communication (Active Listening, Technical Translations to business impacts)

Work Experience

Educational Qualification

- Bachelor’s or Master’s degree in Computer Science, Electronics, or related field

Experience

- Minimum 5 to 6 years of experience in AI/ML & GenAI Testing

📌 Senior QA Automation Engineer (Bengaluru)
🏢 Quest Global
📍 Bengaluru