Roles & Responsibilities
Test and validate AI agents to ensure they perform actions accurately and as designed
Develop and execute test cases for different conversational and task-based AI workflows
Analyze test results, identify bugs or performance issues, and collaborate with the engineering team for fixes
Document testing procedures, outcomes, and recommendations for improvement
Ensure that AI agents handle prompts, contexts, and actions correctly
Contribute to the creation of automated QA processes for AI systems
Ideal Candidate
Robust QA Engineer Profile with AI / GenAI agent testing experience
Mandatory (Experience 1): Must have 2 years of experience in QA, software testing, or AI testing roles.
Mandatory (Experience 2): Must have experience testing and validating AI agents to ensure they perform actions accurately and as designed.
Mandatory (Experience 3): Must be able to develop and execute test cases for conversational and task-based AI workflows.
Mandatory (Experience 4): Must be able to analyze test results, identify bugs or performance issues, and collaborate with engineering for fixes, plus document testing procedures and outcomes.
Mandatory (Tech skill 1): Must have a good understanding of Generative AI, LLM agents, or autonomous AI workflows.
Mandatory (Tech skill 2): Must have an understanding of DeepEval and other AI testing frameworks.
Mandatory (Tech skill 3): Must have manual testing experience and familiarity with automation tools.
Mandatory (Skill): Must have solid problem-solving, analytical, and communication skills.
Mandatory (Collaboration): Must be able to collaborate with developers, prompt engineers, and data scientists.
Preferred (Frameworks): Exposure to LLM frameworks (LangChain, CrewAI, AutoGen, etc.).
Preferred (Programming): Basic programming or scripting knowledge (Java or Python).
Preferred (Metrics): Understanding of AI evaluation metrics and model performance tuning.
Pay: From ₹50,000.00 per month
Advantages
Paid sick time
Paid time off
Application Question(s):
Your Current CTC and Your Notice Period