We are looking an experienced AI Test Engineer to drive quality assurance for AI-powered applications, including LLM models, conversational bots, autonomous agents, and MCP-enabled systems. The ideal candidate will have robust expertise in Playwright with .NET, API/UI automation, AI/LLM validation, and CI/CD automation. The role involves building scalable test frameworks, validating AI responses, testing agent workflows, and ensuring the reliability, security, and performance of GenAI solutions.

Role :AI Test Engineer

Location: PAN India

Experience Range: 5 to 12years

Key Responsibilities:
Design and execute test strategies for LLM-powered applications, AI chatbots, conversational AI, and autonomous agents.
Develop and maintain Playwright (.NET) automation frameworks for UI and API testing.
Validate AI model outputs for:
Accuracy




Relevance
Hallucinations
Consistency
Safety and Guardrails
Test MCP (Model Context Protocol) endpoints and MCP-enabled agent workflows.
Automate API testing and integrate test suites into CI/CD pipelines.
Create test datasets, evaluation metrics, and AI validation frameworks.
Perform regression, functional, integration, and end-to-end testing for AI systems.
Collaborate with developers, data scientists, and product teams to ensure quality delivery.
Monitor and troubleshoot AI system failures, agent interactions, and model behavior.
Implement quality gates within Azure DevOps/GitHub Actions/Jenkins pipelines.

📌 Software Engineer Hyderabad (India)
🏢 Tata Consultancy Services
📍 India