Description About Norstella Norstella is a premier and critical global life sciences data and AI solutions provider dedicated to improving patient access to life-saving therapies. Norstella supports pharmaceutical and biotech companies across the full drug development lifecycle — from pipeline to patient. Our mission is simple: to help our clients bring therapies to market faster and more efficiently, ultimately impacting patient lives. Norstella unites market-leading brands
- Citeline, Evaluate, MMIT, Panalgo, Skipta and The Dedham Group and delivers must-have answers and insights, leveraging AI, for critical strategic, clinical, and commercial decision-making. We help our clients:
• Accelerate the drug development cycle
• Assess competition and bring the right drugs to market
• Make data driven commercial and financial decisions
• Match and recruit patients for clinical trials
• Identify and address barriers to therapies Norstella serves most pharmaceutical and biotech companies around the world, along with regulators like the FDA, and payers. By providing critical proprietary data supporting AI-driven workflows, Norstella helps clients make decisions faster and with greater confidence. Norstella’s investments in AI are transforming how data is consumed and decisions are made, disrupting inefficient legacy workflows and helping the industry become more efficient, cutting-edge, and responsive to patient needs.

Job Description
Every client deliverable and disease data product we ship is real-world data (RWD) that starts as LLM output on messy clinical notes. You decide when that data is fit to leave the building, and you build the system that proves it. Data Ships Only When It Passes Four Tests Test The question What failing looks like Clinically consistent Does each patient's story make clinical sense? Treatment before diagnosis; stage, histology and biomarkers that contradict each other; progression with no supporting evidence Structurally adherent Does the data match its specification? Values outside the code set, wrong data types or date formats, broken keys, inconsistent null and unknown conventions Analytics-ready Can an analyst use it without cleaning it? Silent drift between releases, undocumented missingness, cohorts that shift when a prompt changes Defensible Can we show our work? A value with no source evidence, or a release with no documented checks and sign-off This is not a manual abstraction role. Our extraction agents and LLM judges do the first pass. You design the rules, calibrate the judges, and spend your own attention where automation cannot be trusted. You are part clinical data scientist, part quality engineer and part editor-in-chief, with the authority to say "not yet." Four Altitudes of Quality Most QC stops at the value. You will check quality at four levels, because a patient can have every field valid and still have an impossible story. Altitude What you check Example checks Value Is this value valid? Allowed value sets and ranges, code-set membership, plausibility limits, valid vs. unusual vs. invalid Record Does the extraction faithfully reflect the source note? Source evidence present, negation, temporality, attribution, history vs. current status Patient journey Does the longitudinal story hold together? Order and timing of diagnosis, staging, biomarker testing, treatment lines and progression; contradictions across notes; unexplained gaps Release Is the whole dataset fit for use? Cohort counts and distributions against expectations, drift against the prior release, missingness and denominators, consistency across indications Responsibilities
• The definition of "release-ready." Per-variable specifications, error severity tiers based on clinical and downstream impact, and acceptance thresholds set separately for client deliverables and platform datasets.
• The assurance rule library
- Versioned, testable checks for validity, structural conformance, cross-field logic and patient-journey consistency. You decide what is valid, invalid or merely unusual, and how unknown and not-documented values are treated.
• Agent-assisted QC. Automated checks, LLM-as-judge evaluation and triage, so people look only at disagreements, low-confidence cases and high-severity cases. You calibrate judges against human labels and test for blind spots they share with the extractor.
• Gold standards. Gold sets and annotation guidelines, double review with adjudication on calibration samples, and agreement tracked over time.
• Error intelligence. Every systematic error is classified as an extraction failure, a documentation ambiguity (copy-forward text, templated language, conflicting notes, ambiguous dates) or a specification problem, then routed to the right fix. Findings feed back into prompts, models and specifications.
• Measurement and drift monitoring. Audit sampling plans, per-field precision, recall and confidence intervals, and comparison across indications, model versions, prompt iterations and releases. Regressions are caught before delivery.
• The release gate. A documented pass or fail decision, with a quality scorecard, on every client deliverable and dataset release.
• Analytics-readiness. Data dictionaries,



known limitations and fit-for-use guidance that let analysts and clients use the data correctly without reverse-engineering it.
• Pipeline integration and visibility. You specify and prototype controls, and partner with engineering to embed them in production pipelines and CI/CD. Dashboards show extraction quality by indication to technical and non-technical stakeholders.
• The definition of "release-ready." Per-variable specifications, error severity tiers based on clinical and downstream impact, and acceptance thresholds set separately for client deliverables and platform datasets.
• The assurance rule library. Versioned, testable checks for validity, structural conformance, cross-field logic and patient-journey consistency. You decide what is valid, invalid or merely unusual, and how unknown and not-documented values are treated.
• Agent-assisted QC. Automated checks, LLM-as-judge evaluation and triage, so people look only at disagreements, low-confidence cases and high-severity cases. You calibrate judges against human labels and test for blind spots they share with the extractor.
• Gold standards. Gold sets and annotation guidelines, double review with adjudication on calibration samples, and agreement tracked over time.
• Error intelligence. Every systematic error is classified as an extraction failure, a documentation ambiguity (copy-forward text, templated language, conflicting notes, ambiguous dates) or a specification problem, then routed to the right fix. Findings feed back into prompts, models and specifications.
• Measurement and drift monitoring. Audit sampling plans, per-field precision, recall and confidence intervals, and comparison across indications, model versions, prompt iterations and releases. Regressions are caught before delivery.
• The release gate. A documented pass or fail decision, with a quality scorecard, on every client deliverable and dataset release.
• Analytics-readiness. Data dictionaries, known limitations and fit-for-use guidance that let analysts and clients use the data correctly without reverse-engineering it.
• Pipeline integration and visibility. You specify and prototype controls, and partner with engineering to embed them in production pipelines and CI/CD. Dashboards show extraction quality by indication to technical and non-technical stakeholders.
• Process and people. Documented QA processes and test strategies, plus direction of a small pool of contract clinical reviewers for adjudication when needed. Documented QA processes and test strategies, plus direction of a small pool of contract clinical reviewers for adjudication when needed. Requirements Experience and education
• Master's degree or PhD in epidemiology, pharmacoepidemiology, biomedical or clinical informatics, data science, life sciences or a related field, or equivalent practical experience.
• 4-8 years in RWD evidence generation, data curation science or clinical data quality, including validating extracted or derived variables against source records. Clinical and RWD judgment
• Hands-on experience with EHR data, clinical notes or claims-linked RWD, and sound judgment about what a note can and cannot support: history vs. current status, copy-forward text, problem-list lag, templated language, ambiguous dates.
• Clinical depth in oncology (staging, histology, biomarkers, line of therapy, progression).
• Experience designing QC rules and analyses at both the record level and the patient-journey level, including defining valid vs. invalid values, plausibility limits and date logic.
• Experience preparing data for analysis: you know how commercial, HEOR, market access and clinical development teams use the data, and what makes it unusable to them. Measurement and AI craft
• Validation statistics: sampling design, PPV, sensitivity and specificity, agreement metrics, confidence intervals, and acceptance thresholds tied to downstream use.
• Working knowledge of LLM extraction failure modes in clinical text: negation, temporality, attribution, cross-note reasoning and hallucinated specificity.
• Daily, hands-on use of generative AI tools and agents (Claude or equivalent) in analytical work.
• Proficiency in Python (pandas or similar) and SQL, enough to build checks, analyze outputs at scale and query a data warehouse. Production-grade frameworks are not required. How You Work
• You are a curious skeptic who asks what a value rests on, and you are comfortable saying no, with evidence.
• You think in patients and in distributions, and prefer building a check once to fixing the same error twice.
• You explain error patterns clearly to engineers, scientists and clients. Preferred Qualifications
• Experience calibrating LLM-as-a-judge against human labels, or building LLM evaluation harnesses.
• Experience with data specifications and quality frameworks for analysis-ready data: data dictionaries, schema validation,



data contracts or dataset conventions.
• Background at an RWD or EHR-derived data vendor, or in pharma, biotech or a CRO; or a clinical qualification (MD, PharmD, nursing, biomedical science).
• Familiarity with ICD-10, SNOMED CT, MedDRA or NCI Thesaurus.
• Exposure to rare disease indications.
• Cloud and search

experience:
AWS (including Redshift) and OpenSearch or Elasticsearch.
• Awareness of regulatory expectations for RWE, including fitness-for-use and auditability.
• Experience directing reviewers or contract clinical SME pools, and familiarity with CI/CD and defect tracking. What The First Year Looks Like
• First 90 days: learn and baseline. Audit current deliverables and datasets against the four tests, build the first error taxonomy, and publish a baseline quality scorecard for the priority oncology variables.
• Months 3-6: define and automate. Written acceptance thresholds per variable and use case, agreed with data science and product. Gold sets and annotation guidelines in place. The first versioned rule library runs at value, record and patient-journey level.
• Months 6-12: gate and scale. Agent-assisted QC runs on every client deliverable and dataset release, with human review focused on flagged cases. Regressions across model and prompt versions are caught before delivery, recurring error types decline, and stakeholders see quality by indication on a dashboard. Benefits
• Health Insurance
• Provident Fund
• Life Insurance
• Reimbursement of Certification Expenses
• Gratuity
• 24x7 Health Desk Our guiding principles for success at Norstella 01: Bold, Passionate, Mission-First We have a lofty mission to Smooth Access to Life Saving Therapies and we will get there by being bold and passionate about the mission and our clients. Our clients and the mission in what we are trying to accomplish must be in the forefront of our minds in everything we do. 02: Integrity, Truth, Reality We make promises that we can keep, and goals that push us to new heights. Our integrity offers us the opportunity to learn and improve by being honest about what works and what doesn’t. By being true to the data and producing realistic metrics, we are able to create plans and resources to achieve our goals. 03: Kindness, Empathy, Grace We will empathize with everyone's situation, provide positive and constructive feedback with kindness, and accept opportunities for improvement with grace and gratitude. We use this principle across the organization to collaborate and build lines of open communication. 04: Resilience, Mettle, Perseverance We will persevere – even in difficult and challenging situations. Our ability to recover from missteps and failures in a positive way will help us to be successful in our mission. 05: Humility, Gratitude, Learning We will be true learners by showing humility and gratitude in our work. We recognize that the smartest person in the room is the one who is always listening, learning, and willing to shift their thinking. Norstella is an equal opportunities employer and does not discriminate on the grounds of gender, sexual orientation, marital or civil partner status, pregnancy or maternity, gender reassignment, race, color, nationality, ethnic or national origin, religion or belief, disability or age. Our ethos is to respect and value people’s differences, to help everyone achieve more at work as well as in their personal lives so that they feel proud of the part they play in our success. We believe that all decisions about people at work should be based on the individual’s abilities, skills, performance and behavior and our business requirements. Norstella operates a zero tolerance policy to any form of discrimination, abuse or harassment. Sometimes the best opportunities are hidden by self-doubt. We disqualify ourselves before we have the opportunity to be considered. Regardless of where you came from, how you identify, or the path that led you here
- you are welcome. If you read this

job description
and feel passion and excitement, we’re just as excited about you. All legitimate roles with Norstella will be posted on Norstella’s job board which is located at norstella.com/careers. If a role is not posted on this job board, a candidate should assume the role is not a legitimate role with Norstella. Norstella is not responsible for an application that may be submitted by or through a third-party and candidates should proceed with extreme caution if a third-party approaches them about an open role with Norstella. Norstella will never ask for anything of value or any type of payment during or as part of any recruitment, interview, or pre-hire onboarding process. If you are aware of or have reason to believe a job posting purportedly for a role with Norstella is fraudulent or otherwise not authorized by Norstella, please contact the Company using the following email address:[email protected]. We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.

📌 Clinical Data Assurance Scientist (India)
🏢 Norstella
📍 India