About Us

Deccan AI Experts is a pioneering AI company founded by IIT Bombay and IIM Ahmedabad alumni, with a solid founding team from IITs, NITs, and BITS. We specialize in high-quality human-curated data, AI-first operations, and advanced AI evaluation systems.

About the Role

We are seeking an
Agentic AI Data Annotator (Freelancer)
to support advanced AI evaluation initiatives focused on AI agents, tool use, function calling, reasoning workflows, and human-in-the-loop data annotation.

In this role, you will annotate and evaluate AI agent interactions, tool-use traces, function-calling sequences, reasoning workflows, and multi-turn conversations. Your expertise will help improve AI systems designed for autonomous task execution, agent planning, instruction following, and decision-making.

This position is ideal for professionals with experience in AI data annotation, LLM evaluation, prompt engineering, AI operations, data labeling, software testing, or technical quality assurance.

Responsibilities
- Create deliverables addressing real-world AI agent and tool-use scenarios.




- Annotate and evaluate AI-generated agent conversations, reasoning traces, function calls, and workflow executions.
- Assess AI outputs for correctness, instruction adherence, tool selection, reasoning quality, and task completion.
- Label
tool-use and function-calling traces
, ensuring correct tool selection, parameter usage, execution order, and expected outcomes.
- Perform
single-step action labeling
by identifying whether each agent action is valid, necessary, and aligned with the user's intent.
- Annotate
agent turns
across multi-step workflows, evaluating reasoning flow, context retention, decision quality, and task progression.
- Tag
failure modes
, including hallucinations, incorrect tool selection, missing tool calls, execution errors, reasoning failures, unsafe behavior, and incomplete task execution.
- Perform
instruction-following checks
to verify that AI agents accurately follo

📌 Agentic AI Data Annotator (Pune)
🏢 Deccan AI Experts
📍 Pune