Role Overview:

We are seeking highly qualified Subject Matter Experts (SMEs) across a wide range of academic, research, and industry disciplines to support the evaluation, testing, and improvement of state-of-the-art Large Language Models (LLMs).

In this role, you will leverage your expertise to evaluate AI-generated responses, design domain-specific prompts, identify knowledge gaps, and provide evidence-based feedback to improve the quality and reasoning capabilities of advanced AI systems.

Key Responsibilities

- Create complex, domain-specific prompts to test LLM capabilities.
- Evaluate AI-generated responses for factual accuracy, reasoning, and quality.
- Identify factual errors, logical inconsistencies, and model weaknesses.
- Develop challenging test cases that expose model limitations.
- Provide clear, evidence-based feedback to improve model performance.

Day - to - day task





- Write domain-specific prompts of varying difficulty.
- Compare multiple AI responses and rank them.
- Explain why a response is correct or incorrect using authoritative references.
- Identify ambiguous questions and propose improved prompts.

Minimum Qualifications

- Master's degree or higher in a relevant domain.
- Minimum 3 years of professional, research, or teaching experience.
- Excellent written English communication skills.
- Robust analytical skills and exceptional attention to detail.

Preferred Qualifications

- Experience with LLMs, Generative AI, prompt engineering, or AI evaluation.
- Published research, industry recognition, or teaching experience is a plus.
- Ability to work independently and provide objective, evidence-backed reviews.

📌 Domain Expert - STEM roles (India)
🏢 Codefeast
📍 India