AI Evaluation Specialist - RLHF, SFT & MCP (Remote, Immediate Joiners)

OpsLabel is expanding its expert network, and we have an immediate need for AI evaluation specialists. We need your judgment on frontier AI projects spanning RLHF, SFT, and MCP (agent tool-use) evaluation, helping train and stress-test how well models respond, reason, and use tools.

• Remote (India preferred)
• Freelance / Contract, long-term project work
• Open to strong technical minds: ML, engineering, linguistics, or a STEM domain. Advanced degree a plus, not required. What matters is sound judgment on what a good AI answer looks like.

Must have:
• Solid judgment on the quality, correctness, and safety of AI model outputs
• Clear written reasoning in English
• Comfort working to detailed rubrics and guidelines
• A technical or analytical background (ML, software, data, linguistics, or a STEM field)

Nice to have:
• Hands-on experience with RLHF, preference ranking, or reward modeling
• SFT data creation: instruction-response pairs or chain-of-thought
• Familiarity with MCP, function or tool calling,



or agent workflows
• Programming or prompt-engineering background

What you'll do:
• Rank and compare model responses to build high-quality preference data
• Write and review instruction-response and chain-of-thought examples
• Evaluate agent tool-use: tool-calling accuracy and multi-step workflows
• Catch the subtle errors and reasoning failures generic reviewers miss
• Deliver high-quality work across multiple sprints with direct feedback from senior reviewers

Why this role matters:
Your judgment directly shapes how the next generation of AI responds, reasons, and uses tools. This is high-impact, flexible work for experts who want their depth to count, not faceless annotation work.

Engagement model:
• Flexible hours, work on your own schedule
• Long-term engagements on frontier AI projects
• Multiple delivery sprints across 3 to 6+ months

How to apply:
Submit your CV along with a short no

📌 AI Evaluation Specialist (India)
🏢 OpsLabel
📍 India