Lead research and production of coding-focused benchmarks built around real AI failure cases.

What you'll do

- Build multiple benchmarks, with a repeatable task-design, validation and review process that keeps quality high.
- Lead our current benchmarking projects with AI labs.
- Study agent runs, group the failures and turn them into objectively graded tasks.
- Define what makes an evaluation useful: correctness, objective grading and difficulty.
- Guard against answer leakage and gaming, and measure genuine improvement.
- Guide the researchers who write and review tasks.

Non-negotiable

You have personally worked on building one or more benchmarks. Sole ownership is not required, but your own contribution must be clear, with an artifact or public work we can inspect.

Robust skills

Coding-agent evals such as Terminal-Bench and SWE-bench style tasks; experimental judgment (what a benchmark does and does not measure); reproducible pipelines; quality control across many outputs

📌 Head of Research, Benchmarks (Mumbai)
🏢 Adzzat
📍 Mumbai