The Opportunity

As a QE - AI Test Engineer, you will play a key role in validating and assuring the quality of AI-powered solutions, including Narrow AI, Generative AI, and Agentic AI systems. You will work closely with cross-functional teams to evaluate AI models, validate data quality, perform statistical analysis, and ensure AI applications meet functional, performance, and business requirements. This role provides an exciting opportunity to work at the forefront of AI innovation while applying quality engineering practices to build reliable, scalable, and trustworthy AI solutions.

Your Key Responsibilities

- Validate AI models and systems across Narrow AI, Generative AI, and Agentic AI implementations to ensure quality, reliability, and performance.

- Perform data validation activities, including data collection, data generation, data augmentation, exploratory data analysis, and assessment of data quality, bias, and privacy considerations.

- Evaluate AI model outputs against functional, business, and performance metrics to identify defects, inconsistencies, and improvement opportunities.

- Design and execute test scenarios for AI-based applications using established software testing principles and practices.

- Apply AI-specific testing approaches such as pairwise testing, metamorphic testing, back-to-back testing, bias testing, and drift testing.

- Utilize Python-based frameworks and automation tools to improve testing efficiency and coverage.

- Collaborate with developers, data scientists, product owners, and stakeholders to identify, track, and resolve quality issues throughout the development lifecycle.

- Support validation of Large Language Models (LLMs)



and Agentic AI systems using established evaluation methodologies and frameworks.

- Contribute to continuous improvement initiatives by recommending enhancements to testing processes, methodologies, and quality standards.

- Work effectively within Agile teams to support delivery of high-quality AI solutions.

Skills and attributes for success

- Proven experience in testing AI models and systems

- Strong understanding of software testing principles, methodologies, and quality engineering practices.

- Foundational knowledge of AI/ML concepts, algorithms, and model validation techniques.

- Understanding of Large Language Models (LLMs), Generative AI, and Agentic AI systems.

- Experience with AI testing techniques such as pairwise testing, metamorphic testing, bias testing, drift testing, and back-to-back testing.

- Proficiency in Python programming for test development, validation, and automation activities.

- Experience with test automation tools such as Selenium, Playwright, PyTest, or similar frameworks.

- Knowledge of cloud technologies and platforms including Microsoft Azure, AWS, or Google Cloud Platform.

- Exposure to GitHub Copilot and AI-assisted development tools.

- Robust analytical, troubleshooting, and problem-solving capabilities.

- Excellent communication and stakeholder management skills.

- Ability to collaborate effectively in Agile delivery environments and multidisciplinary teams.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

📌 Testing - AI Test Professional (Bengaluru)
🏢 EY
📍 Bengaluru