- Test and validate AI agents to ensure they perform actions accurately and as designed
- Develop and execute test cases for different conversational and task-based AI workflows
- Analyze test results, identify bugs or performance issues, and collaborate with the engineering team for fixes
- Document testing procedures, outcomes, and recommendations for improvement
- Ensure that AI agents handle prompts, contexts, and actions correctly
- Contribute to the creation of automated QA processes for AI systems
Ideal Candidate
- Strong QA Engineer Profile with AI / GenAI agent testing experience
- Mandatory (Experience 1): Must have 2+ years of experience in QA, software testing, or AI testing roles.
- Mandatory (Experience 2): Must have experience testing and validating AI agents to ensure they perform actions accurately and as designed.
- Mandatory (Experience 3): Must be able to develop and execute test cases for conversational and task-based AI workflows.
- Mandatory (Experience 4): Must be able to analyze test results, identify bugs or performance issues, and collaborate with engineering for fixes, plus document testing procedures and outcomes.
- Mandatory (Tech skill 1): Must have a valuable understanding of Generative AI, LLM agents, or autonomous AI workflows.
- Mandatory (Tech skill 2): Must have an understanding of DeepEval and other AI testing frameworks.
- Mandatory (Tech skill 3): Must have manual testing experience and familiarity with automation tools.
- Mandatory (Skill): Must have strong problem-solving, analytical, and communication skills.
- Mandatory (Collaboration): Must be able to collaborate with developers, prompt engineers, and data scientists.