Domain Expert / Subject Matter Expert (SME)

Experience:
3–6 Years

Job Type:
Contract / Project-Based

Work Mode:
Remote

About the Role

We are looking for
Domain Experts / Subject Matter Experts (SMEs)
from diverse academic, research, and industry backgrounds to support the
evaluation, testing, and improvement of advanced Large Language Models (LLMs)
.

The ideal candidate will leverage strong subject-matter expertise and analytical skills to evaluate AI-generated responses, identify knowledge gaps, design challenging prompts, and provide evidence-based feedback to improve AI model performance.

? Work Shift

7:30 PM – 12:30 AM IST

*Remaining 4 hours are flexible.*

Key Responsibilities
- Create complex, domain-specific prompts to evaluate LLM capabilities.
- Evaluate AI-generated responses for
factual accuracy, reasoning, relevance, and overall quality
.
- Identify factual errors, logical inconsistencies, ambiguities, and knowledge gaps.
- Develop challenging test cases that expose AI model limitations.
- Compare and rank multiple AI-generated responses.




- Provide clear, objective, and evidence-based feedback.
- Explain why a response is correct or incorrect using authoritative references.
- Identify ambiguous questions and improve prompts for better evaluation.
- Support the development of high-quality evaluation datasets and benchmarks.

Required Qualifications
- Master’s degree or higher
in a relevant academic or professional domain.
- 3–6 years of professional, research, or teaching experience
.
- Solid analytical and critical-thinking abilities.
- Excellent written English communication skills.
- Exceptional attention to detail.
- Ability to independently evaluate complex information and provide objective feedback.

Preferred Qualifications
- Experience with
LLMs, Generative AI, Prompt Engineering, or AI Evaluation
.
- Experience evaluating AI-generated content or datasets.
- Published research, teaching experience, or industry recognition.
- Ability to w

📌 Language Specialist (India)
🏢 Sourcebae
📍 India