AI / Quality Assurance Engineer

Experience: 4-7
Years

Role Type: QA Engineering / Test
Automation / AI Quality

Primary Stack: Azure | .NET APIs |
Next.js | Postman | GitHub | Docker | AI/LLM Testing

Position Overview

We are seeking a hands-on AI / Quality Assurance Engineer to
test modern cloud applications and AI-enabled solutions built on Microsoft
Azure. The ideal candidate will have solid experience in functional, API,
integration, regression, and automated testing, along with an understanding of
AI/LLM applications. The role will validate traditional application
functionality as well as AI agents, RAG, MCP integrations, model responses,
security, and end-to-end AI workflows.

Key Responsibilities

Application & API
Testing

· Develop and execute functional, regression,
integration, API, and end-to-end test scenarios.

· Test C#/.NET ASP.NET Core APIs using Postman and
automated testing frameworks.

· Validate Next.js/TypeScript application
workflows.

· Create test plans, test cases, test data, and
acceptance criteria.

· Identify, document, track,



and validate defects
through resolution.

· Validate authentication, authorization, error
handling, API contracts, and enterprise integrations.

· Support UAT, release validation, and
production-readiness testing.

AI & Agent Testing

· Test Azure OpenAI/OpenAI applications, AI
agents, multi-step workflows, and tool/function calling.

· Test Model Context Protocol (MCP) client/server
interactions, tool execution, authorization, and failure handling.

· Validate RAG pipelines including ingestion,
retrieval, filtering, grounding, and citations.

· Test structured AI outputs and response schemas.

· Create AI evaluation datasets and regression
scenarios.

· Evaluate response accuracy, groundedness,
relevance, citation quality, latency, and reliability.

· Test negative scenarios including prompt
injection, unauthorized tool access, hallucination, and sensitive-data
exposure.

Automation & DevOps

· Develop automat

📌 AI Quality Assurance Engineer (Pune)
🏢 Palo Alto Labs
📍 Pune