AI agents are increasingly being asked to do real scientific work, not just answer questions about it. A fast-growing AI evaluation team is building expert-level benchmarks that test whether agents can take a professional chemistry task from raw inputs to a verified result, entirely inside a terminal.

We're looking for practicing chemists to design those challenges. You'll take a real, paid task from your own recent work and turn it into a self-contained Terminal-Bench problem: the input files and data, the software workplace it runs in, a working reference solution, and tests that objectively prove whether an answer is right. If you've ever thought "a model would get this wrong, and here's exactly how I'd catch it," this role is built for you.

This is remote, flexible, task-based work. Compensation is paid per approved task, and full details are provided in the application questions.

What You'll Do

- Turn a real chemistry task from your professional work into an expert-level challenge that an AI agent must solve end to end in a terminal.
- Define the task prompt, input files, data, and software environment so the challenge is realistic, self-contained, and reproducible.
- Write a complete, working reference solution in Python, R, C++, or the command line, showing that a skilled practitioner can solve the task in roughly 60–75 minutes.
- Build tests and verification checks that objectively prove correctness, such as an identified structure, an assigned spectrum,



or a computed value within numerical tolerance.
- Document how a mistake would be caught, along with the tools, data sources, and domain rules involved, so reviewers can validate the scientific accuracy.
- Iterate on tasks based on reviewer feedback, tuning difficulty so each task stays expert-level, unambiguous, and fair.

What You Bring
- A PhD in chemistry or a chemistry subfield (completed or at an advanced stage), or 3+ years of professional research experience, backed by publications, a thesis, or lab output.
- Hands-on chemistry work within the last 3 years, with the ability to name the specific systems, software, data, and rules you used.
- Experience running analyses or simulations in code (Python, R, C++, Julia, or MATLAB) or working directly with domain software.
- A clear sense of what a checkable outcome looks like in your field, such as a confirmed structure or a number that must land within tolerance.
- Comfort on the command line and with files and data formats such as CSV and JSON, plus working familiarity with APIs or databases.
- Availability to start in early November 2026 and to complete tasks on a per-task basis.

Nice to Have

- First-author publications with released code or data.
- Contributions to open-source or scientific software.
- Experience teaching, mentoring, or peer reviewing in your subfield.
- Working knowledge of Docker, Git, and automated testing (e.g., pytest).
- Experience running jobs on HPC clusters.

📌 Computational Science Chemistry Expert for AI Benchmarking (India)
🏢 Toptal
📍 India