We are looking an experienced AI Test Engineer to drive quality assurance for AI-powered applications, including LLM models, conversational bots, autonomous agents, and MCP-enabled systems. The ideal candidate will have solid expertise in Playwright with .NET, API/UI automation, AI/LLM validation, and CI/CD automation. The role involves building scalable test frameworks, validating AI responses, testing agent workflows, and ensuring the reliability, security, and performance of GenAI solutions.

Role :AI Test Engineer

Location: PAN India

Experience Range: 5 to 12years

Key Responsibilities:

- Design and execute test strategies for LLM-powered applications, AI chatbots, conversational AI, and autonomous agents.
- Develop and maintain Playwright (.NET) automation frameworks for UI and API testing.
- Validate AI model outputs for:
- Accuracy
- Relevance




- Hallucinations
- Consistency
- Safety and Guardrails
- Test MCP (Model Context Protocol) endpoints and MCP-enabled agent workflows.
- Automate API testing and integrate test suites into CI/CD pipelines.
- Create test datasets, evaluation metrics, and AI validation frameworks.
- Perform regression, functional, integration, and end-to-end testing for AI systems.
- Collaborate with developers, data scientists, and product teams to ensure quality delivery.
- Monitor and troubleshoot AI system failures, agent interactions, and model behavior.
- Implement quality gates within Azure DevOps/GitHub Actions/Jenkins pipelines.

📌 Software Engineer (Hyderabad)
🏢 Tata Consultancy Services
📍 Hyderabad