Job Summary

Perform functional, regression, integration, and API testing for AI-powered applications. Test LLM, GenAI, RAG, and agentic AI workflows across diverse user scenarios. Evaluate LLM responses for accuracy, relevance, consistency, groundedness, and completeness.

Validate RAG pipelines, including retrieval quality, context relevance, and response grounding. Test AI agents for tool selection, execution, workflow completion, and failure handling. Create AI test datasets, evaluation scenarios, edge cases, and adversarial test cases.

Automate high-value test scenarios using Selenium, Playwright, Cypress, or similar frameworks. Perform API testing using REST APIs and Postman, and integrate automated tests into CI/CD pipelines. Use AI testing and evaluation tools such as Ragas, DeepEval, Promptfoo, and LangSmith for AI quality assessment.

Analyze defects, perform root-cause analysis, validate fixes, and support performance, scalability, and reliability testing using Python or other scripting languages.

Responsibilities

- Perform functional, regression, integration, and API testing for AI-powered applications.

- Test LLM, GenAI, RAG, and agentic AI workflows across diverse user scenarios.





- Evaluate LLM responses for accuracy, relevance, consistency, groundedness, and completeness.

- Validate RAG pipelines, including retrieval quality, context relevance, and response grounding.

- Test AI agents for tool selection, execution, workflow completion, and failure handling.

- Create AI test datasets, evaluation scenarios, edge cases, and adversarial test cases.

- Automate high-value test scenarios using Selenium, Playwright, Cypress, or similar frameworks.

- Perform API testing using REST APIs and Postman, and integrate automated tests into CI/CD pipelines.

- Use AI testing and evaluation tools such as Ragas, DeepEval, Promptfoo, and LangSmith for AI quality assessment.

- Analyze defects, perform root-cause analysis, validate fixes, and support performance, scalability, and reliability testing using Python or other scripting languages.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

📌 Quality Assurance Engineer (Hyderabad)
🏢 Knowledgesprint Technologies
📍 Hyderabad