Role - QA – Agentic AI Designation - Engineer - Testing Experience (in years) - 2 - 5 years (total experience - expected for the role) Education - e.g., Bachelor’s degree, preferably in Computer Science, Electrical Engineering, Physics, Math or any other related discipline.

Summary The Agentic AI QA Analyst is responsible for validating the quality, reliability, safety, and correctness of Agentic AI systems. This includes testing autonomous agents, multi-step reasoning flows, tool orchestration, and human-in-the-loop controls. The role requires solid analytical skills to evaluate non-deterministic AI behavior. Roles & Responsibilities

● Test agentic AI workflows involving planning, reasoning, action, and reflection loops

● Validate autonomous decision-making paths and goal completion behavior

● Test prompt chaining, context persistence, and memory handling

● Validate tool calling (APIs, databases, search, RPA, internal services)

● Design tests for hallucinations, partial failures, retries, and recovery logic

● Validate human-in-the-loop checkpoints, approvals, overrides, and escalation paths

● Perform functional, exploratory, regression, and scenario-based testing

● Document defects with reproducible prompts, agent state, tools invoked, and outputs

● Support risk-based testing and compliance requirements (CSA / GxP where applicable)

● Under the Sycamore Product and coordinate with different teams to get the work done Essential Experience

Technical Skills – Agentic AI Testing





● Understanding of Agentic AI architectures (Planner–Executor, ReAct, AutoGPT-style agents)

● Experience testing multi-agent systems and agent-to-agent communication

● Prompt testing and prompt-chain validation techniques

● Testing non-deterministic outputs using evaluation heuristics and acceptance bands

● Experience validating LLM tool calling and function calling

● API testing (REST/GraphQL) for agent-integrated services

● Ability to inspect logs, traces, and agent execution graphs

● Familiarity with vector databases, embeddings, and retrieval-augmented generation (RAG)

● Understanding of model limitations, hallucination patterns, and AI failure modes

Deep-Dive Testing Experience (Agentic AI)

● Testing planning accuracy and goal decomposition across multi-step tasks

● Validating agent memory (short-term vs long-term) and context carryover

● Testing tool misuse, incorrect tool selection, and fallback mechanisms

● Evaluating output consistency across multiple runs

● Testing edge cases such as ambiguous prompts, conflicting goals, and incomplete data

● Validating explainability and reasoning traces where available

● Assessing bias, unsafe outputs, and guardrail effectiveness Desired Experience

● Exposure to automation (Python, Robot Framework, API automation)

● Experience with AI evaluation frameworks or red-teaming

● Experience in regulated environments (GxP, CSA, 21 CFR Part 11)

📌 QA Agentic AI (India)
🏢 Sycamore Informatics
📍 India