Key Responsibilities

- Design and own the AI evaluation framework: benchmark construction, automated and human-in-the-loop eval pipelines.
- Lead red-teaming and adversarial testing across deployed models: prompt injection, jailbreaking,

edge case identification.

- Define AI safety standards, responsible AI policies, and child-safe content guardrails for

deployment at scale.

- Build and maintain evaluation tooling: RAGAS, LangSmith, PromptBench, EleutherAI lmevaluation-

harness.

- Conduct systematic bias detection and fairness measurement across outputs, especially for

regional and linguistic variation.

- Manage AI risk assessments and translate safety findings into actionable model or prompt

changes.

- Establish AI governance documentation: model cards, audit trails, safety reports for internal and

regulatory stakeholders.

- Embed evaluation and safety checkpoints into the model release pipeline in collaboration with

engineering.

- Track and apply evolving regulatory requirements: EU AI Act, NIST AI RMF, India AI policy

frameworks.

- Measure and report model accuracy, grounding fidelity,



and response quality across deployed AI

systems through systematic evaluation.

- Build and maintain knowledge of AI governance and compliance standards, operationalising

responsible AI practices into daily engineering workflows.

Must-Have Skills

- Strong experience in AI/LLM safety, evaluation, and risk assessment
- Expertise in designing and executing model evaluation frameworks and test methodologies
- Experience in red-teaming, adversarial testing, and identifying model vulnerabilities
- Solid understanding of bias, fairness, hallucination, and content safety in AI systems
- Experience measuring model accuracy, grounding, and response quality
- Knowledge of AI governance, compliance, and responsible AI practices
- Familiarity with frameworks such as NIST AI RMF and AI regulatory standards; understanding of student psychology and age-appropriate AI output design
- Strong analytical and problem-solving skills with experience working on production AI systems

📌 AI Safety & Evaluation Lead (Uttar Pradesh)
🏢 Smart Source
📍 Uttar Pradesh