AI Evaluation Engineer
Location: Gurugram, India
Seniority: Mid-level (3-6 years)
Purpose: Build and execute FNZ's AI evaluation framework and tooling, working under the guidance of the AI Evaluations Team Lead to assess quality, safety, robustness, and operational suitability of AI solutions before release.
Key Responsibilities:
- Build, maintain, and evolve core components of FNZ's AI evaluation framework, including test structures, scoring approaches, reusable evaluation patterns, and supporting documentation.
- Develop and improve evaluation tooling, harnesses, automation, and CI/CD integrations used to run repeatable assessments across AI agents and workflows.
- Execute evaluations across FNZ's six-pillar framework, including Task Performance, Safety & Compliance, Efficiency, Groundedness & Reasoning, Robustness, and Suitability.
- Create and maintain test datasets, golden sets, rubrics, and scoring criteria that reflect expected agent behaviour and business requirements.
- Build and run test suites covering baseline behaviour, edge cases, failure modes,
and adversarial scenarios for AI agents and workflows.
- Assess multi-step agentic workflows, including planning, tool use, execution quality, recovery from errors, and adherence to controls
- Verify groundedness and output quality by checking responses against source content, expected reasoning patterns, and policy constraints.
- Document evaluation findings with clear evidence, communicate issues to AI solution teams, and support remediation and re-testing.
- Develop deeper expertise in one or two evaluation domains while remaining effective as a generalist across the wider framework.
Skills and Experience:
- 3-6 years in software engineering, test engineering, AI/ML development, or data science.
- Solid programming skills with hands-on experience building test automation, evaluation tooling, or developer productivity tooling; Python or .NET background required, with .NET preferred
- Practical experienc
📌 AI Evaluation Engineer (Gurugram)
🏢 FNZ
📍 Gurugram