Apply on Kit Job: kitjob.in/job/44fw31

Job Description

Role: QA Engineer – Generative AI / Responsible AI Testing

Experience: 4–7 Years

Location: Open to any Orion Innovation Office in India ( Chennai/Pune/ Mumbai/Kochi/Coimbatore/Hyderabad)

Employment Type: FTE

Role Overview

We are seeking a detail-oriented QA Engineer with experience in Generative AI and Responsible AI testing to ensure the quality, safety, and reliability of AI-powered applications, including LLM-driven chatbots and intelligent systems. The role will focus on functional, adversarial, and ethical validation of GenAI solutions while supporting automation and performance testing initiatives.

Key Responsibilities

GenAI & Responsible AI Testing

- Design and execute prompt-based test cases covering functionality, bias, safety, and edge scenarios.
- Craft adversarial prompts to stress-test LLM behavior and identify vulnerabilities.
- Validate hallucination resistance, output consistency, and prompt reliability.
- Evaluate AI systems for bias, fairness, transparency, and ethical compliance.
- Test content moderation and safety filters for GenAI applications.
- Perform end-to-end QA for GenAI chatbot conversational flows.

Functional & System Testing

- Design test scenarios, test cases, and regression suites.
- Conduct System Integration Testing (SIT), Smoke Testing, Regression Testing, and Bug Testing.
- Execute manual and automated test cycles across releases.
- Maintain defect logs, triage defects, and track resolution.

Automation & Performance Testing

- Develop and maintain automation scripts using Python-based frameworks.
- Build automated tests using Pytest / Selenium / Robot Framework.




- Support load and performance testing to assess system scalability.

BI & Data Validation Testing

- Validate BI dashboards, reports, and visualizations.
- Ensure data accuracy, reporting reliability, and performance across tools like Tableau and Power BI.
- Support data storytelling and analytics validation.

Agile & DevOps Collaboration

- Participate in Sprint Planning, Backlog Grooming, and Daily Stand-ups.
- Collaborate with Developers, Product Owners, and Analysts to streamline QA processes.
- Support build deployments across QA, UAT, and Production environments.
- Provide feature demos and stakeholder updates.

Required Skills & Competencies

Testing Expertise

- Software Testing Life Cycle (STLC) & SDLC
- Test case design & execution
- Defect management & triaging
- Regression & integration testing

GenAI / AI Testing

- LLM testing & validation
- Prompt engineering for testing
- Responsible AI principles
- Bias & fairness evaluation

Automation & Tools

- Python
- Pytest
- Selenium / Robot Framework
- LangChain (preferred)

BI & Data Tools

- Tableau
- Power BI
- Advanced Excel

Project & QA Tools

- Jira
- Confluence
- Azure DevOps

Qualifications

- Bachelor’s Degree in Computer Science, Engineering, or related field.
- ISTQB CTFL certification preferred.

Nice to Have

- Knowledge of Machine Learning / Deep Learning concepts.
- Experience with GenAI governance frameworks.
- Exposure to statistical analysis and data science fundamentals.

Soft Skills

- Robust analytical & problem-solving skills
- Excellent communication & stakeholder management
- Attention to detail
- Ability to work in Agile teams

Apply on Kit Job: kitjob.in/job/44fw31

📌 Gen AI Test Engineer (Nellore)
🏢 Orion Innovation
📍 Nellore