AI Evaluation Lead (Part Time Contract, Engagement Based)
Dataclap Digital Ventures | Bangalore, Chennai or Coimbatore (remote first, occasional visits to Coimbatore)
About us
Dataclap is an AI data services company based in Coimbatore, working with AI companies across North America and Europe on data annotation, human in the loop workflows, RLHF and MLOps. We're building a dedicated AI evaluation practice covering LLMs, AI agents, computer vision and audio models, and we're looking for experienced practitioners to lead client engagements and mentor our engineering team.
The role
This is a part time, engagement based contract role, not full time employment. You'll lead AI evaluation projects for our clients, typically 8 to 15 hours a week per engagement. You'll design the evaluation approach, guide a team of junior engineers who do the hands on execution, review their work, and own the quality of what we deliver. You'll also help us turn every engagement into reusable methods, so the practice gets stronger with each project.
What you'll do
Scope evaluation projects with clients:



understand the model, the use case, and what "valuable" looks like for them
Design evaluation frameworks, including metrics, test sets, rubrics, and pass/fail criteria
Build or guide the building of golden datasets and evaluation harnesses
Design and calibrate LLM as a judge setups, and know when human evaluation is needed instead
Plan red teaming, safety and robustness testing where relevant
Lead and mentor 2 to 5 junior engineers per engagement through regular reviews and working sessions
Review results, catch flawed methodology, and present findings to client stakeholders
Document each engagement as a playbook, rubric set or template the team can reuse
What we're looking for
6+ years in machine learning, data science or applied AI, with at least 2 years focused on model evaluation, benchmarking or quality assurance of ML systems
Hands on experience evaluating LLM applications: RAG systems, ch

📌 AI Evaluation Lead (Coimbatore)
🏢 DATACLAP DIGITAL
📍 Coimbatore