AI Evaluations & Red Team Lead

Experience: 7 to 12 years

Location: Pune, India

Skills: AI evaluations, red teaming, LLM-based agents, RAG architectures, agentic AI systems, programming, evaluation methodologies, probabilistic AI systems, financial services, healthcare

Purpose
Lead FNZ's unified AI evaluations and red teaming capability, ensuring all AI agent solutions meet rigorous safety, performance, and compliance standards before production deployment.

Key Responsibilities
• Define and evolve AI evaluation strategy aligned to FNZ's six-pillar framework (Task Performance, Safety & Compliance, Efficiency, Groundedness & Reasoning, Robustness, Suitability)
• Establish evaluation standards, methodologies, and scoring criteria integrated into FNZ's SDLC as mandatory release gates
• Represent evaluations function in AI Governance Committee, providing risk assessments and release recommendations
• Build, mentor, and manage team of 4 evaluation specialists across generalist and specialist roles
• Design and execute complex evaluations for high-risk AI agents; lead red teaming exercises for critical deployments
• Communicate evaluation findings to technical and non-technical stakeholders; influence product roadmaps

Skills and Experience
• 7-12 years in AI/ML engineering, quality assurance,



or security testing with 3+ years in evaluation/red teaming
• Deep understanding of LLM-based agents, RAG architectures, and agentic AI systems (not model training)
• Solid programming background; with real world experience developing evaluation solutions for agents.
• Proven ability to design evaluation methodologies for probabilistic AI systems
• Experience in regulated industries (financial services, healthcare) with compliance awareness
• Leadership experience managing cross-functional teams and influencing stakeholders

About FNZ
FNZ is committed to opening up wealth so that everyone, everywhere can invest in their future on their terms. We know the foundation to do that already exists in the wealth management industry, but complexity holds firms back. We created wealth’s growth platform to help. We provide a global, end-to-end wealth management platform that integrates modern technology with business and investment operations. All in a regulated financial institution. We partner with the world’s leading financial institutions, with over US$2.4 trillion in assets on platform (AoP). Together with our clients, we empower nearly 30 million people across all wealth segments to invest in their future.

📌 AI Evaluations & Red Team Lead (Pune)
🏢 FNZ Group
📍 Pune