Home/Jobs/Fullstack AI Quality Engineer – Applied & Agentic AI Systems

Fullstack AI Quality Engineer – Applied & Agentic AI Systems

Roche

Hyderabad

7+ years

2 days ago

$56.6K–71.1K/yr

Full time

Onsite

Skills Required LLM

RAG

Agentic AI

Gen AI

Agentic workflow validation

LangChain

AI evaluation frameworks

Python

Pytest allure

TypeScript

Playwright

Jest

Cypress

React

Description Roche is seeking an AI Quality Engineer specialized in testing Generative and non-Generative AI applications and agentic systems. The role involves full-stack quality assurance including manual and automated testing across Python and TypeScript stacks, focusing on AI and LLM-based systems.

Company: Roche

Role: AI Quality Engineer

Location: Hyderabad, India

Experience

- 7+ years of experience in software testing
- At least 1 year focused on genAI and/or agentic applications
- Experience working in regulated environments
- Experience working with Agile development teams

Qualification

- B.Sc. in Computer Science, Software Engineering, or equivalent
- B.Eng. in Computer Science, Software Engineering, or equivalent
- M.Sc. in Computer Science, Software Engineering, or equivalent
- M.Eng. in Computer Science, Software Engineering, or equivalent
- English communication skills at C1+ level

Responsibilities

- Design and implement testing strategies for LLM-based and agentic AI systems
- Conduct manual exploratory testing and automated UI/API/MCP testing
- Validate classical machine learning and generative AI behaviors
- Develop and maintain quality assurance frameworks and compliance documentation
- Identify and test edge cases, biases, and fairness in AI model outputs
- Collaborate on security and performance testing of AI applications
- Integrate automated testing suites into CI/CD pipelines
- Curate and manage evaluation datasets for benchmarking
- Leverage observability tools to monitor AI system performance and feedback

Additional Responsibilities





- Co-create and maintain AI-specific testing frameworks
- Document testing strategies, test cases, and guidelines specific to AI
- Work in a regulated environment with qualified infrastructure and validated applications
- Understand GxP vs non-GxP products and CSV and ITSM processes
- Stay current with latest AI and GenAI testing methodologies and best practices
- Communicate findings and recommendations clearly
- Work hours structured to overlap with Central European Time golden hours

Nice To Have

- Practical experience testing applications on AWS cloud platform
- Experience with monitoring, log analysis, and performance metrics tracking
- Experience with Roche ALM tools such as Jira and GitHub
- Use of AI coding assistants and autonomous agents to accelerate development and testing
- Conduct code reviews for human-written and AI-generated code
- Understanding of AI quality assurance standards in regulated industries

More Skills GitHub Actions, CI/CD pipelines, RAGAS, DeepEval, Promptfoo, LangSmith, Arize Phoenix, AWS, Grafana, Jira, GitHub, AI evaluation tools, AI/LLM testing, Automated UI/API testing, Machine learning validation, Generative AI evaluation, Agentic AI systems, Security testing methodologies

Prepare for this role

Recommended resources to build the skills for this position. Sponsored.

Top 10 RAG Quick-Prep Questions

Zenaique A shorter RAG interview prep set for quick review.

Top 20 LLM System Design Questions (Quick)

Zenaique A focused LLM system design interview question set.

Top 50 LLM System Design Interview Questions

Zenaique

Curated LLM system design questions from Zenaique.

More LLM jobs

AI/LLM Safety Engineer

Propio Language Services

Leawood

Today

AI/ML Engineer 2

Day And Zimmermann

United States

Today

Software/AI developer

Rackner

United States

Today

Conversational AI Engineer

Bright Vision Technologies

United States

Today

Sr Lead Software Engineer - Agentic AI Solutions

JPMorgan Chase

Mumbai

Today

Senior Technical Consultant — AI Engineering & Agentic Salesforce Delivery

Salesforce

Bengaluru

Today

📌 Fullstack AI Quality Engineer – Applied & Agentic AI Systems (Hyderabad)
🏢 Top Gen AI Jobs
📍 Hyderabad