1. Role Summary

Oncoshot turns unstructured clinical documents (clinical notes, pathology, radiology and genomic
reports) into a structured, research-grade record of each institution’s cancer population:
diagnoses, histology, TNM and overall stage, biomarkers, treatments and disease status. Hospitals
and research partners rely on that record only as far as its quality can be shown. The Data
Scientist (Evaluation & Data Quality) owns the science of measuring it: how gold-standard data is
sampled and annotated, how accuracy is estimated and compared between releases, how far model
confidence can be trusted, and how complete and consistent the record is at each institution.

This is a hands-on, statistics-driven role. The engineers build and ship the pipelines; the Data
Scientist defines how quality is measured, what counts as a real improvement and where the bar
sits. Patient-derived data never leaves the institution, so analyses run inside each hospital’s
setting or on de-identified and aggregated results. To see how we measure extraction quality,




look at our open-source evaluation framework, llmvalidate
(https://github.com/Oncoshot/llm-validation-framework), which this role works with and extends.
2. Key Responsibilities
- Design the gold standard: Sample evaluation data stratified by document type, institution and
rare but clinically important values, size each sample for the precision required, and decide
when it must be refreshed.
- Run annotation as a measurement process: With the clinical validation team, write annotation
guidelines, measure inter-annotator agreement, adjudicate, and turn recurring disagreements into
sharper field definitions.
- Own the evaluation methodology: Specify how each field is scored, including multi-valued and
hierarchical fields such as staging and biomarkers; report precision, recall, F1 and F2 with
confidence intervals; and define the statistical tests that let release gates separate a real
change from noise.
- Calibrate confi

📌 Data Scientist (Thane)
🏢 Oncoshot India
📍 Thane