AI Data Quality Lead – Robotics & Video Annotation | Remote

About the Role

We are looking for an experienced AI Data Quality Lead to own the quality standards for egocentric training data used to develop and evaluate physical-AI and robotics models.

You will be responsible for building and maintaining the golden sets, annotation SOPs, rubrics, calibration systems, quality frameworks, and audit processes that ensure high-quality ground-truth data. You will also work closely with engineering and product teams to develop annotation workflows for hand tracking, object identification, and action labeling.

This is a ground-floor opportunity to build a 0→1 data quality system at a fast-growing, well-funded startup. You will not simply inherit an existing process—you will help establish the quality standards, workflows, and systems from the ground up.

Key Responsibilities

- Own the end-to-end quality framework for egocentric training data, including:

- Golden and benchmark datasets

- Acceptance criteria

- Frame-level tolerance thresholds

- Sampling strategies

- Audit and quality-control loops

- Author, maintain, and version annotation SOPs, rubrics, and edge-case guidelines as project specifications evolve.

- Translate ambiguous requirements into precise, actionable instructions for annotators and reviewers.

- Establish and manage annotator training, certification, and calibration programs.

- Run calibration sessions and adjudicate disagreements between annotators.

- Measure and report quality using metrics such as:

- Inter-Annotator Agreement (IAA)

- Kappa/alpha

- Tolerance-based agreement for temporal boundaries

- Acceptance/rejection rates

- Rework rates

- Cost per accepted hour

- Identify root causes of quality issues and determine whether they result from guideline ambiguity, annotation errors, or workflow issues.

- Design human-in-the-loop workflows where AI/model-generated labels are reviewed, corrected, and escalated by human annotators.

- Audit auto-generated labels and improve annotation throughput without compromising quality.





- Partner with engineering and product teams to develop new annotation workflows for:

- Hand tracking

- Object identification

- Action labeling

- Define label schemas, annotation-tool requirements, QC dashboards, and pilot-validation processes.

- Validate annotation workflows through pilot programs before scaling them into production.

- Work with ML teams to understand how annotation quality impacts model performance.

- Contribute to tooling and workflow improvements where required.

Required Skills & Qualifications

- 5+ years of hands-on video annotation experience, including at least 1+ year working with egocentric/first-person video for robotics or embodied AI.

- 3+ years of experience in a lead or QA capacity, including annotator calibration, disagreement adjudication, and ownership of annotation guidelines.

- Demonstrated experience building 0→1 annotation programs and taking them from pilot through production.

- Strong command of annotation-quality methodologies, including:

- IAA frameworks

- Kappa/alpha

- Gold-set creation

- Sampling strategies

- Tolerance thresholds

- Acceptance criteria

- Ability to design quality systems and explain the practical difference between quality targets such as 95% and 99% accuracy.

- Proven experience authoring, rather than simply following, annotation SOPs and rubrics.

- Ability to translate ambiguous specifications into clear and actionable annotation guidelines.

- Hands-on experience with at least 2 professional annotation platforms, such as:

- CVAT

- Labelbox

- Encord

- V7

- Label Studio

- Equivalent annotation tools

- Experience configuring label schemas and QA workflows within annotation platforms.

- Experience auditing model-generated or automatically generated labels.





- Experience designing human-in-the-loop annotation workflows.

- Strong written and verbal English communication skills.

- Strong analytical skills and proficiency with Excel and/or Google Sheets for quality reporting.

- Robust understanding of ML concepts and how annotation quality affects model performance.

- Data-driven and process-oriented mindset with strong ownership.

- Comfortable working in ambiguous, fast-moving, startup environments.

- Strong ability to work collaboratively with cross-functional and remote teams.

Preferred / Bonus Qualifications

- Background in robotics, mechanical engineering, mechatronics, or computer vision engineering.

- Experience working at a leading robotics company, AI/robotics startup, or similar project.

- Experience managing distributed or global annotator workforces or external annotation vendors.

- Experience with multimodal annotation, including:

- 3D/depth data

- Joint pose

- Grasp outcome classification

- Experience training or fine-tuning autolabeling models.

- Experience partnering closely with ML teams responsible for automated labeling.

- Experience building annotation tooling or working closely with annotation-tooling engineering teams.

- Experience with physical AI, embodied AI, computer vision, or robotics datasets.

Why This Role?

The quality bar is the product.

The ground-truth data produced through this role will become a benchmark against which physical-AI models are measured. You will have direct ownership of the systems that determine whether that data meets the required quality standards.

This is a ground-floor opportunity at a fast-growing, well-funded startup where you will build the quality system from scratch rather than inherit an established process.

Role Highlights

- Role: AI Data Quality Lead – Robotics & Video Annotation

- Work Type: Remote

- Engagement: [Contract / Full-Time – specify as applicable]

- Focus: AI Training Data, Robotics, Embodied AI & Video Annotation

- Experience: 5+ years relevant experience

📌 AI Video Annotation/Quality Lead - Remote (India)
🏢 Yo HR Consultancy
📍 India