The role
You'll own the quality bar for shift's egocentric training data: the golden sets, SOPs, and calibration system that every annotator, reviewer and labeling model is measured against. You'll also work with engineering to stand up recent annotation workflows in quality control, hand tracking, object identification, and action labelling.

What you'll do
Own the quality bar end-to-end: golden/benchmark sets, acceptance criteria, frame-level tolerance thresholds, and the audit loops that hold them
Author and version annotation SOPs, rubrics, and edge-case rules as the client spec evolves
Measure and report quality: IAA (kappa/alpha, tolerance-based agreement on temporal boundaries), acceptance/rejection, rework, cost-per-accepted-hour — and root-cause disagreements to guideline ambiguity vs. annotator error
Set up audit frameworks for quality control and drive human-in-the-loop operations to detect bad quality vs good quality data
Stand up annotator training and certification, run calibration sessions, and design human-in-the-loop workflows where models pre-label,



and humans review, correct, and escalate
Partner with engineering and product on new workflows (hand tracking, object identification, action labeling): label schemas, tool requirements, QC dashboards, and pilot validation before production scaling

What we're looking for
7+ years hands-on video annotation and quality control
4+ years in a lead or QA capacity: calibrating annotators, adjudicating disagreements, owning guidelines others work to
Built 0→1 annotation programs and taken them from pilot to production
Experience in integrating autolabeling and model-based annotation into human workflows; building human-in-the-loop pipelines to raise throughput without sacrificing quality.
Strong cross-functional partnership with product, engineering, and research/ML.
Authored (not just followed) SOPs and rubrics, with deep command of quality methodology: IAA frameworks, gold-set creation, sampling strategies, tol

📌 Annotation Quality Program Manager (India)
🏢 Shift
📍 India