Role & responsibilities
Design and own the AI evaluation framework: benchmark construction, automated and human-in

the-loop eval pipelines.
Lead red-teaming and adversarial testing across deployed models: prompt injection, jailbreaking,

edge case identification.
Define AI safety standards, responsible AI policies, and child-secure content guardrails for

deployment at scale.
Build and maintain evaluation tooling: RAGAS, LangSmith, PromptBench, EleutherAI lm

evaluation-harness.
Conduct systematic bias detection and fairness measurement across outputs, especially for

regional and linguistic variation.
Manage AI risk assessments and translate safety findings into actionable model or prompt

changes.
Establish AI governance documentation: model cards, audit trails, safety reports for internal and

regulatory stakeholders.
Embed evaluation and safety checkpoints into the model release pipeline in collaboration with

engineering.




Track and apply evolving regulatory requirements: EU AI Act, NIST AI RMF, India AI policy

frameworks.
Measure and report model accuracy, grounding fidelity, and response quality across deployed AI

systems through systematic evaluation.
Build and maintain knowledge of AI governance and compliance standards operationalising

responsible AI practices into daily engineering workflows.

Must-Have Skills
Solid experience in AI/LLM safety, evaluation, and risk assessment
Expertise in designing and executing model evaluation frameworks and test methodologies
Experience in red-teaming, adversarial testing, and identifying model vulnerabilities
Solid understanding of bias, fairness, hallucination, and content safety in AI systems
Experience measuring model accuracy, grounding, and response quality

📌 Ai Safety & Evaluation Lead Noida (India)
🏢 LAKSH HUMAN RESOURCE
📍 India