Head of AI Evaluation & Reliability Engineering
Location: Versatile / Hybrid
Reports To: Head of Engineering

Role Mission
Build and scale Codvo’s AI Evaluation & Reliability Engineering capability as a core engineering function supporting the design, validation, and continuous improvement of enterprise AI systems in production.
You will architect the frameworks, tooling, benchmark assets, and operational processes required to ensure AI systems deployed by Codvo and its customers meet enterprise standards for reliability, safety, performance, and governance.
This role is deeply embedded within engineering and serves as the quality and reliability backbone for Codvo’s AI platform and delivery organization.

Why This Role Matters
As AI systems move from pilots to business-critical workflows, reliability and evaluation become core engineering disciplines—not optional afterthoughts.
Codvo is building the infrastructure and operational rigor required to ensure every AI deployment is measurable, governed, and production-ready.

Core Responsibilities
Engineering Ownership




- Build Codvo’s AI Evaluation & Reliability Engineering function as a core platform/engineering capability.
- Define engineering standards for AI evaluation, testing, release gating, and runtime monitoring.
- Integrate evaluation/reliability frameworks into Codvo’s engineering and delivery lifecycle.

Evaluation Architecture
- Design reusable evaluation frameworks for:
- LLM / multimodal quality
- RAG grounding / evidence fidelity
- Agent reasoning / decision quality
- Tool / workflow execution success
- Safety / policy / compliance adherence
- Cost / latency / production economics

Benchmark Infrastructure
- Build benchmark packs, golden datasets, and regression suites for priority enterprise workflows.
- Define benchmark coverage and versioning standards.
- Establish processes for edge-case capture and benchmark expansion.

Runtime Reliability Systems
- Design systems/processes for:
- Runtime drift / deg

📌 Head of AI Evaluation (Pune)
🏢 Codvo.ai
📍 Pune