We are seeking a highly motivated and experienced AI Quality Engineer to join our Retail and Wealth Risk Engineering team under the Enterprise Risk Technology platform. This role spans the full spectrum of up-to-date AI quality engineering — from Agentic AI flow testing and RAG pipeline validation to AI safety, test automation, and performance & reliability engineering.
You will be the quality pillar for complex autonomous AI systems, ensuring they are safe, accurate, explainable, resilient, and production-ready at scale. This is a high-impact, highly technical role that requires both depth in AI/ML and breadth across testing disciplines.

Responsibilities
Agentic AI Testing
• Design and execute end-to-end test strategies for Agentic AI pipelines, including single-agent and multi-agent workflows.
• Validate agent reasoning, planning, and decision-making chains (e.g., ReAct, Chain-of-Thought, Plan-and-Execute, Reflexion).
• Test tool-use correctness — ensuring agents invoke the right tools, with correct parameters, at the right time.
• Evaluate agent memory systems (short-term,



long-term, episodic) for accuracy and context retention across sessions.
• Validate agent handoff and delegation logic in multi-agent orchestration frameworks (e.g., AutoGen, CrewAI, LangGraph).
• Test termination conditions, loop detection, and infinite loop prevention in autonomous agent loops.

RAG (Retrieval-Augmented Generation) Testing
• Design comprehensive test strategies for end-to-end RAG pipelines — covering ingestion, chunking, embedding, retrieval, reranking, and generation stages.
• Validate retrieval accuracy and relevance — ensuring the correct context chunks are retrieved for a given query.
• Test embedding model quality and vector similarity thresholds across different document corpora.
• Evaluate faithfulness, groundedness, and answer relevance of generated responses using frameworks like RAGAS, TruLens, DeepEval.
• Test chunking strategies (fixed, semantic, hierarchical) for th

📌 AI Quality Engineer, Safety and RAG (Pune)
🏢 Citi
📍 Pune