Evaluate AI-generated responses against quality standards and help improve AI training, benchmarking, and evaluation datasets.
Key Responsibilities
- Evaluate AI responses using standardized rubrics
- Assess factuality, grounding, relevance, completeness & helpfulness
- Identify hallucinations and unsupported claims
- Evaluate safety & instruction following
- Provide detailed rating justifications
- Handle geographic, maps, local search & POI-related information
- Participate in calibration and quality improvement activities
Required Skills
- Understanding of Generative AI / LLM behavior
- Prompt-response evaluation & hallucination detection
- Strong analytical and critical-thinking skills
- Geographic reasoning & navigation concepts
- Excellent written English & reading comprehension
- Attention to detail and evidence-based decision making
Eligibility
- Experience: 0–3 years
- Background in Data Annotation, Content Evaluation, Technical Writing or related fields
- AI/LLM evaluation experience preferred
- Bachelor’s degree in Linguistics, English, Computer Science, Humanities or related field
- Equivalent practical experience may also be considered
Positive to Know: Basic SQL, annotation platforms, Jira, documentation tools & AI evaluation platforms. Pay: ₹18,000.00 - ₹50,000.00 per month