Evaluate AI-generated responses against quality standards and help improve AI training, benchmarking, and evaluation datasets.
Key Responsibilities
Evaluate AI responses using standardized rubrics
Assess factuality, grounding, relevance, completeness & helpfulness
Identify hallucinations and unsupported claims
Evaluate safety & instruction following
Provide detailed rating justifications
Handle geographic, maps, local search & POI-related information
Participate in calibration and quality improvement activities
Required Skills
Understanding of Generative AI / LLM behavior
Prompt-response evaluation & hallucination detection
Robust analytical and critical-thinking skills
Geographic reasoning & navigation concepts
Excellent written English & reading comprehension
Attention to detail and evidence-based decision making
Eligibility
Experience: 0–3 years
Background in Data Annotation, Content Evaluation, Technical Writing or related fields
AI/LLM evaluation experience preferred
Bachelor’s degree in Linguistics, English, Computer Science, Humanities or related field
Equivalent practical experience may also be considered
Positive to Know: Basic SQL, annotation platforms, Jira, documentation tools & AI evaluation platforms. Pay: ₹18,000.00 - ₹50,000.00 per month