We are seeking highly analytical and detail-oriented AI Evaluation Analysts to support high impact artificial intelligence research projects.

In this role, you will critically evaluate AI-generated responses against defined criteria and provide clear, structured written feedback on their quality, accuracy, reasoning, relevance and presentation.

The work requires more than identifying obvious errors. You will be expected to recognise nuance, interpret implicit meaning, identify gaps or weaknesses in reasoning, and explain your assessments through precise, evidence-based written rationales. You may also evaluate the visual quality and structure of AI-generated presentation materials, assessing areas such as layout, alignment, hierarchy, spacing, use of containers, consistency and overall design coherence.

This role is well suited to independent thinkers who are comfortable making critical judgments, applying detailed guidelines consistently, and defending their conclusions in writing.

Key Responsibilities

- Review and evaluate AI-generated written responses against detailed evaluation criteria and project guidelines.
- Assess responses for accuracy, relevance, completeness, clarity, logical reasoning and adherence to instructions.
- Identify subtle errors, inconsistencies, unsupported assumptions, omissions and weaknesses in reasoning.
- Evaluate whether responses appropriately understand and address both explicit and implicit user intent.
- Compare AI-generated outputs and determine which response performs better against defined quality standards.
- Write clear, concise and well-evidenced rationales explaining evaluation decisions.
- Assess AI-generated presentation slides and other visual outputs for layout, alignment, spacing, hierarchy, symmetry, consistency and overall design quality.
- Identify presentation issues such as poorly anchored design elements, inappropriate positioning, overcrowded text containers and inconsistent visual balance.
- Apply structured scoring frameworks and evaluation rubrics accurately and consistently.




- Follow detailed task-specific instructions while exercising independent judgment where required.
- Maintain a high standard of accuracy and attention to detail across repetitive or complex evaluation tasks.
- Provide candid and objective assessments, including critical feedback where an AI-generated response does not meet the required standard.
- Complete evaluation assignments independently and within required project timelines.
- Maintain confidentiality and comply with all project, data-handling and research requirements.

Basic Qualifications

- Bachelor's degree from a top-500 globally ranked university preferred.
- Native-level English fluency.
- Robust analytical and critical-thinking skills.
- Excellent written English and the ability to communicate complex judgments clearly and precisely.
- Strong attention to detail and the ability to identify subtle differences in meaning, reasoning and quality.
- Ability to work independently and follow detailed written instructions.
- Comfortable working within structured evaluation frameworks and applying scoring criteria consistently.

Required Skills

- Critical Reading & Reasoning - Ability to critically analyse written material and identify nuance, implicit meaning, logical inconsistencies, unsupported conclusions and gaps in reasoning.
- Written Evaluation - Ability to produce clear, precise and well-evidenced written rationales that explain why an output is strong or weak rather than relying on surface-level observations.
- Visual & Presentation Evaluation - Ability to recognise issues relating to slide composition, alignment, spacing, visual hierarchy, symmetry, text placement,



container sizing and overall presentation quality.
- Judgment & Objectivity - Ability to make consistent and honest assessments based on the evidence presented, including giving critical evaluations when warranted.
- Attention to Detail - Ability to identify small but meaningful errors or inconsistencies and accurately apply detailed evaluation guidelines.
- Instruction Following - Ability to interpret and follow complex task requirements, rubrics and project-specific instructions without deviating from the required methodology.
- Independent Working - Ability to manage assigned tasks independently, maintain concentration and consistently deliver high-quality evaluations.
- Eligibility:

1. Native English fluency is required.
2. Candidates must be able to demonstrate excellent written English and critical reasoning ability
3. Candidates must be comfortable completing assessment tasks as part of the selection process
4. Candidates must be able to perform the work entirely without the use of AI writing or generative AI tools
5. All evaluations, rationales and written submissions must represent the candidate's own independent analysis and writing

- Ideal Candidate - The ideal candidate is intellectually curious, highly observant and naturally critical when reviewing information. You should be comfortable questioning whether an answer genuinely makes sense, identifying weaknesses that may not be immediately obvious, and explaining your conclusions convincingly.

You will be equally comfortable identifying reasoning problems in written responses and recognising visual issues in generated materials — for example, when design components appear unanchored, when text does not sit appropriately within its containers, or when the positioning of elements makes a slide feel asymmetrical or visually unbalanced. Above all, you will bring accuracy, consistency, independent judgment and exceptionally strong attention to detail to every evaluation.

📌 AI Evaluation Analyst (India)
🏢 Nuram
📍 India