We’re supporting a team that builds expert-level evaluations to test how well AI agents handle real professional work from the command line. We’re looking for music theorists, composers, arrangers, and transcribers who also code. You’ll turn the kind of music work you’ve been paid to do into challenges an AI agent must solve end to end. Each challenge includes the files, a working solution, and an automated check that proves the result is correct.

This is a task-based role: you’re paid a fixed rate per accepted task, on your own schedule. Compensation details are in the application questions.

What You’ll Do

- Design expert-level Terminal-Bench challenges modeled on real, paid music work you’ve done, such as arranging, transcription, harmonization, or engraving. Use original or public-domain material, not client-owned scores.
- Build the task environment, including scores, MusicXML or MIDI assets, and any supporting files, so it is realistic and fully reproducible.
- Write a working reference solution using Python, music21, or command-line tools. The task should be solvable by a skilled expert in about 60–75 minutes.
- Build automated tests that objectively verify the result. They should encode craft rules such as voice leading, harmony, and notation, so that errors are caught reliably.
- Package each task with Docker and Git where appropriate, with transparent instructions, so it runs consistently for reviewers.




- Revise tasks based on reviewer feedback, tightening difficulty, ambiguity, and edge cases so each one stays expert-level, objectively gradable, and hard for current AI agents.

What You Bring

- 3+ years of professional work in music theory, composition, arranging, or transcription, or a music degree plus recent paid work in the field, or a PhD with active research in the area.
- Hands-on music work within the last 3 years. You should be able to name the tools, systems, data, and rules you used.
- The ability to state the objective rules of your craft, including voice leading, harmony, and notation, and to explain how a violation would be detected.
- Working knowledge of machine-readable music formats such as MusicXML, MIDI, and music21.
- Fluency in Python or another language and the command line, plus comfort with files, CSV/JSON, APIs, or databases.
- Availability to start in early November 2026.

Nice to Have

- Docker experience for packaging reproducible environments
- Git and experience writing automated tests
- Experience teaching harmony or ear training
- Professional transcription or engraving work in MuseScore, Sibelius, or Finale

Why This Role

- A rare chance to combine serious musicianship with code.
- Your expertise defines how AI agents are measured on real music work.
- Fully remote, flexible hours, with a fixed rate per accepted task.

📌 Music Theory Expert – AI Benchmark Task Designer (India)
🏢 Toptal
📍 India