Job Title: AI Policy Reviewer
Location: Remote (Worldwide)
Job Summary: The AI Policy Reviewer is responsible for evaluating AI-generated and user-generated content to ensure compliance with internal governance standards, regulatory requirements, and responsible AI principles. This role plays a key part in safeguarding model integrity by reviewing outputs for safety risks, bias, misinformation, harmful content, and policy violations, while ensuring consistent enforcement of AI usage guidelines.
Responsibilities:
· Review and score AI-generated responses against detailed policy rubrics. Assess outputs for safety, truthfulness, fairness, and alignment with community guidelines.
· Act as a quality assurance checkpoint for automated systems. Identify instances where the AI misinterprets policy (e.g., being over-sensitive and censoring benign content, or under-sensitive and allowing harmful content).
· Handle complex “edge cases” where policy application is ambiguous. Make nuanced judgement calls regarding context, satire, or emerging risks that the AI model struggles to process.




· Analyze and review data to identify systematic flaws in the AI’s reasoning. Report patterns of bias, hallucination, or policy gaps to the Product and Engineering teams.
· Collaborate with Policy teams to test and refine evaluation rubrics. Provide feedback on whether current policies are “teachable” to AI models or if they require human-only judgement.
· Participate in adversarial testing (red teaming) by attempting to “jailbreak” the model or provoke unsafe responses to identify vulnerabilities before launch.
· Work closely with Machine Learning Engineers to explain the “why” behind your ratings, helping them adjust model behavior.
· Write high-quality examples (prompts and ideal responses) that’s serve as “golden sets” for training the AI on how to handle difficult policy scenarios.
Requirements:
· Minimum of 3 years of skilled experience in Trust & Safety Operations, Content Policy, Ri

📌 A.I Policy Reviewer (India)
🏢 Odixcity Consulting
📍 India