Programmatic Support for evaluating Large Language Model
• Review actual queries received from FLWs during chatbot pilot deployment and develop accurate, comprehensive, and evidence-based ideal answers using the approved knowledge base.
• Interpret government guidelines, health manuals, and programme documents to formulate clear, contextually appropriate answers aligned with ASHA roles and responsibilities.
• Ensure ground-truth answers capture all relevant recommendations, conditions, exceptions, timelines, referral criteria, and follow-up instructions, wherever applicable.
• Identify the relevant source documents, sections, pages, and paragraphs supporting each answer to ensure traceability.
• Identify queries that are ambiguous, out of scope, or cannot be answered adequately using the available knowledge base, and document these for expert review.
• Create ideal answers for variations in query phrasing, including colloquial language, multilingual queries, and code-mixed queries, where applicable.
• Ensure answers are factually accurate, complete, easy to understand, and appropriate for frontline health workers.
• Identify gaps, inconsistencies, and ambiguities in the knowledge base and flag them to the relevant public health and technical teams.
• Digitise government guidelines and manuals into structured Markdown, preserving headings, tables, flowcharts, and other relevant content.




• Maintain structured records of field queries, ideal answers, supporting references, and issues requiring clarification.
• Occasionally may include digitisation of government guidelines, manuals, and other source documents into Markdown format, preserving their structure, content, tables, and flowcharts to support effective integration into the chatbot's knowledge base. Requirements
• Bachelor's or master's degree in Public Health, Nursing, Medicine, Life Sciences, or a related health discipline.
• 1–2 years of relevant experience in public health, health programme implementation, medical content development, document digitisation, or a related field is mandatory.
• Strong reading comprehension and the ability to interpret government health guidelines and translate them into accurate, evidence-based answers.
• Transparent understanding of with India's public health system and frontline health workers, particularly ASHAs, ANMs, and the National Health Mission is expected.
• Excellent attention to detail and the ability to identify inconsistencies, missing information, and unsupported interpretations.
• Strong written communication skills, with the ability to present health information clearly and logically.
• Proficiency in Microsoft Word, Excel, Google Docs, and Google Sheets.

📌 Data Expert (New Delhi)
🏢 Wadhwani AI LEHS
📍 New Delhi