- Test and validate AI agents to ensure they perform actions accurately and as designed
- Develop and execute test cases for different conversational and task-based AI workflows
- Analyze test results, identify bugs or performance issues, and collaborate with the engineering team for fixes
- Document testing procedures, outcomes, and recommendations for improvement
- Ensure that AI agents handle prompts, contexts, and actions correctly
- Contribute to the creation of automated QA processes for AI systems
Ideal Candidate
- Strong QA Engineer Profile with AI / GenAI agent testing experience
- Mandatory (Experience 1): Must have 2 years of experience in QA, software testing, or AI testing roles.
- Mandatory (Experience 2): Must have experience testing and validating AI agents to ensure they perform actions accurately and as designed.
- Mandatory (Experience 3): Must be able to develop and execute test cases for conversational and task-based AI workflows.
- Mandatory (Experience 4): Must be able to analyze test results, identify bugs or performance issues, and collaborate with engineering for fixes, plus document testing procedures and outcomes.
- Mandatory (Tech skill 1): Must have a good understanding of Generative AI, LLM agents, or autonomous AI workflows.
- Mandatory (Tech skill 2): Must have an understanding of DeepEval and other AI testing frameworks.
- Mandatory (Tech skill 3): Must have manual testing experience and familiarity with automation tools.
- Mandatory (Skill): Must have solid problem-solving, analytical, and communication skills.
- Mandatory (Collaboration): Must be able to collaborate with developers, prompt engineers, and data scientists.
- Preferred (Frameworks): Exposure to LLM frameworks (LangChain, CrewAI, AutoGen, etc.).
- Preferred (Programming): Basic programming or scripting knowledge (Java or Python).
- Preferred (Metrics): Understanding of AI evaluation metrics and model performance tuning.
Pay: From ₹50,000.00 per month
Benefits
- Paid sick time
- Paid time off
Application Question(s):
- Your Current CTC and Your Notice Period