Role : The GenAI Specialist Rating

Experience - 0-3 yrs

Locations: Hyderabad

Job Type : Full Time

***************************************************************************

IMMEDIATE JOINERS ALERT!

We're looking for candidates who can join immediately.

If you're available, please send your CV via WhatsApp only to: (phone hidden)

Please note: No calls will be entertained.

****************************************************************************

Role Summary

The GenAI Specialist Rating evaluates AI-generated responses against defined quality rubrics to determine whether responses meet gold-standard expectations. Specialists assess factual accuracy, grounding, completeness, helpfulness, safety, and user satisfaction while providing clear rationale for every decision.

Their work directly contributes to high-quality evaluation datasets used for model training, benchmarking, and calibration.

Key Responsibilities

- Evaluate AI-generated responses using standardized evaluation rubrics
- Assess factual accuracy of geographic and local information
- Verify that responses are properly grounded in available map and place data
- Evaluate:

Factuality

Grounding

Helpfulness

Completeness

Relevance

Clarity

Safety

Instruction following

- Identify hallucinations and unsupported claims
- Distinguish between critical and minor defects
- Provide detailed justification for ratings
- Escalate ambiguous or policy-sensitive cases
- Maintain consistency across evaluation tasks
- Participate in calibration exercises




- Contribute feedback to improve evaluation guidelines

Core Skills

Generative AI Evaluation

- Understanding of LLM behavior
- Prompt-response evaluation
- Hallucination detection
- Response quality assessment

Analytical Thinking

- Critical reasoning
- Evidence-based decision making
- Attention to detail
- Pattern recognition

Maps Knowledge

- Geographic reasoning
- Navigation concepts
- POIs (Points of Interest)
- Local search behavior
- Route interpretation

Language Skills

- Excellent written English
- Reading comprehension
- Ability to interpret nuanced prompts

Secondary Skills

- Search verification techniques
- Knowledge of local businesses
- Travel and navigation terminology
- Familiarity with Maps products
- Basic data annotation experience
- Spreadsheet proficiency

Ancillary Skills [Valuable to know]

- SQL (basic)
- Annotation platforms
- Jira
- Documentation tools
- AI evaluation platforms

Qualifications

- Work Experience: 03 years in data annotation, content evaluation, technical writing, or a related field. Experience with AI/LLM evaluation or editing is preferred.
- Educational Qualifications: Bachelors degree in Linguistics, English, Computer Science, Humanities, or a related field, or equivalent practical experience.

Success Measures

- Rating consistency
- Calibration accuracy
- Low disagreement rate
- Annotation quality
- Productivity
- Low audit defect rate

Budget : 3.5-4.5 LPA(25% Max Hike on current CTC)

📌 GenAI Specialist - Rating | 0-3 yrs | Hyderabad
🏢 Gryphon Hytech Services
📍 Hyderabad