Remote (Work From Anywhere)
Experience: 3+ Years
About the Role
We are seeking a highly analytical and detail-oriented LLM Analys t to evaluate, optimize, and improve the performance of Large Language Models (LLMs) and Generative AI systems. The ideal candidate will have experience in AI evaluation, prompt engineering, content quality analysis, research, data analysis, or AI operations.
As an LLM Analyst, you will play a key role in assessing model outputs, identifying performance gaps, analyzing user interactions, and providing actionable insights that enhance AI quality, accuracy, safety, and overall user experience.
Key Responsibilities
LLM Evaluation & Analysis
Evaluate responses generated by Large Language Models (LLMs) across diverse domains and use cases.
Assess outputs for accuracy, relevance, completeness, consistency, reasoning quality, and instruction adherence.
Identify hallucinations, factual inaccuracies, biases, and safety concerns.
Analyze model strengths, weaknesses, and performance trends.
Prompt Analysis & Optimization




Review and optimize prompts to improve AI-generated outcomes.
Conduct prompt testing across various scenarios and user journeys.
Develop prompt evaluation methodologies and best practices.
Recommend improvements to increase response quality and reliability.
AI Performance Monitoring
Track and analyze AI performance metrics and quality indicators.
Identify recurring issues and opportunities for model improvement.
Conduct root cause analysis on AI failures and inconsistencies.
Support benchmarking initiatives across different AI models and versions.
User Interaction & Behavioral Analysis
Analyze user interactions and feedback to understand model effectiveness.
Identify patterns in user behavior and common failure scenarios.
Recommend enhancements based on user needs and business objectives.
Support initiatives focused on improving user satisfaction and engagement.
Quality Assurance & Validation
Design and execute AI testing and valida

📌 Llm Analyst (Mysuru)
🏢 AcrossTek™
📍 Mysuru