Role & responsibilities

- Design and own the AI evaluation framework: benchmark construction, automated and human-in

the-loop eval pipelines.
- Lead red-teaming and adversarial testing across deployed models: prompt injection, jailbreaking,

edge case identification.
- Define AI safety standards, responsible AI policies, and child-secure content guardrails for

deployment at scale.
- Build and maintain evaluation tooling: RAGAS, LangSmith, PromptBench, EleutherAI lm

evaluation-harness.
- Conduct systematic bias detection and fairness measurement across outputs, especially for

regional and linguistic variation.
- Manage AI risk assessments and translate safety findings into actionable model or prompt

changes.
- Establish AI governance documentation: model cards, audit trails, safety reports for internal and

regulatory stakeholders.
- Embed evaluation and safety checkpoints into the model release pipeline in collaboration with

engineering.




- Track and apply evolving regulatory requirements: EU AI Act, NIST AI RMF, India AI policy

frameworks.
- Measure and report model accuracy, grounding fidelity, and response quality across deployed AI

systems through systematic evaluation.
- Build and maintain knowledge of AI governance and compliance standards operationalising

responsible AI practices into daily engineering workflows.

Must-Have Skills

- Strong experience in AI/LLM safety, evaluation, and risk assessment
- Expertise in designing and executing model evaluation frameworks and test methodologies
- Experience in red-teaming, adversarial testing, and identifying model vulnerabilities
- Strong understanding of bias, fairness, hallucination, and content safety in AI systems
- Experience measuring model accuracy, grounding, and response quality

📌 AI Safety & Evaluation Lead - Noida
🏢 LAKSH HUMAN RESOURCE
📍 Noida