Head of AI Evaluation & Reliability Engineering
Location: Versatile / Hybrid
Reports To: Head of Engineering

Role Mission
Build and scale Codvo’s AI Evaluation & Reliability Engineering capability as a core engineering function supporting the design, validation, and continuous improvement of enterprise AI systems in production.
You will architect the frameworks, tooling, benchmark assets, and operational processes required to ensure AI systems deployed by Codvo and its customers meet enterprise standards for reliability, safety, performance, and governance.
This role is deeply embedded within engineering and serves as the quality and reliability backbone for Codvo’s AI platform and delivery organization.

Why This Role Matters
As AI systems move from pilots to business-critical workflows, reliability and evaluation become core engineering disciplines—not optional afterthoughts.
Codvo is building the infrastructure and operational rigor required to ensure every AI deployment is measurable, governed, and production-ready.

Core Responsibilities
Engineering Ownership




Build Codvo’s AI Evaluation & Reliability Engineering function as a core platform/engineering capability.
Define engineering standards for AI evaluation, testing, release gating, and runtime monitoring.
Integrate evaluation/reliability frameworks into Codvo’s engineering and delivery lifecycle.

Evaluation Architecture
Design reusable evaluation frameworks for:
LLM / multimodal quality
RAG grounding / evidence fidelity
Agent reasoning / decision quality
Tool / workflow execution success
Safety / policy / compliance adherence
Cost / latency / production economics

Benchmark Infrastructure
Build benchmark packs, golden datasets, and regression suites for priority enterprise workflows.
Define benchmark coverage and versioning standards.
Establish processes for edge-case capture and benchmark expansion.

Runtime Reliability Systems
Design systems/processes for:
Runtime drift / deg

📌 Head Of Ai Evaluation Pune
🏢 Codvo.ai
📍 Pune