We are looking for an AI QA & Prompt Analyst to ensure the quality, reliability, and performance of our AI models and systems. You will design test strategies for LLM-based applications and work closely with our AI engineering teams.
What You’ll Do
Design and execute test plans for LLM-based applications and AI systems
Develop and optimize prompts for various AI use cases
Identify and document AI model limitations, biases, and failure modes
Create automated testing frameworks for AI quality assurance
Collaborate with AI engineers to improve model performance
Must Have Skills
3+ years of QA experience with at least 1 year in AI/ML testing
Robust understanding of LLMs and prompt engineering principles
Experience with Python for test automation
Familiarity with AI evaluation metrics and methodologies
Excellent analytical and problem-solving skills
Nice-to-Have
Experience with MLflow, LangChain, or similar frameworks
Knowledge of RAG architectures and vector databases
Background in NLP or data science
APPLY NOW ↗
AI - QA & Prompt Analyst
Role Overview
We are looking for an AI QA & Prompt Analyst to ensure the quality, reliability, and performance of our AI models and systems. You will design test strategies for LLM-based applications and work closely with our AI engineering teams.
What You’ll Do
Design and execute test plans for LLM-based applications and AI systems
Develop and optimize prompts for various AI use cases
Identify and document AI model limitations, biases, and failure modes
Create automated testing frameworks for AI quality assurance
Collaborate with AI engineers to improve model performance
Must Have Skills
3+ years of QA experience with at least 1 year in AI/ML testing
Strong understanding of LLMs and prompt engineering principles
Experience with Python for test automation
Familiarity with AI evaluation metrics and methodologies
Excellent analytical and problem-solving skills
Nice-to-Have
Experienc
📌 2. AI - QA & Prompt Analyst (Hyderabad)
🏢 Trinity AI Boston
📍 Hyderabad
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.