Home/Jobs/Fullstack AI Quality Engineer – Applied & Agentic AI Systems

Fullstack AI Quality Engineer – Applied & Agentic AI Systems

Roche

Hyderabad

7+ years

2 days ago

$56.6K–71.1K/yr

Full-time

Onsite

Skills Required LLM

RAG

Agentic AI

Gen AI

LangChain

RAGAS

Agentic Workflow Validation

Promptfoo

DeepEval

Golden Datasets

LangSmith

Arize Phoenix

Python

Pytest allure

Description Roche is seeking an AI Quality Engineer to test Generative and non-Generative AI applications and agentic systems, ensuring reliability, fairness, and effectiveness of AI-powered solutions in healthcare.

Company: Roche

Role: AI Quality Engineer

Location: Hyderabad, India

Experience

- 7+ years of experience in software testing
- At least 1 year focused on genAI and/or agentic applications
- Experience working in regulated environments with qualified infrastructure and validated applications
- Experience working with Agile development teams

Qualification

- B.Sc., B.Eng., M.Sc., M.Eng. in Computer Science, Software Engineering, or equivalent degree
- Strong background in testing methodologies
- English communication skills at C1+ level

Responsibilities

- Design and implement testing strategies for LLM-based and agentic AI systems
- Perform manual exploratory testing and automated UI/API testing
- Validate classical machine learning and generative AI behaviors
- Develop and maintain quality assurance frameworks and compliance documentation
- Identify and test edge cases, biases, and fairness in AI model outputs
- Collaborate on security and performance testing of AI applications
- Create and maintain comprehensive test documentation and guidelines
- Integrate automated testing suites within CI/CD pipelines
- Curate and manage evaluation datasets for benchmarking
- Leverage observability and feedback loops to monitor AI system performance

Additional Responsibilities

- Work in a highly regulated workplace ensuring safety, compliance, and robustness of AI solutions




- Use critical thinking and problem-solving for AI prompt testing and data distribution analysis
- Proactively identify potential issues and suggest improvements
- Communicate findings and recommendations clearly
- Align working hours to overlap with Central European Time for collaboration

Nice To Have

- Practical experience testing applications on cloud platforms such as AWS
- Experience with monitoring, observability tools, and log analysis
- Familiarity with Roche mandatory ALM tools like Jira and GitHub
- Use of AI coding assistants and autonomous agents to accelerate development and testing
- Conduct code reviews for human-written and AI-generated code
- Understanding of security testing methodologies for AI applications
- Knowledge of regulatory compliance and AI quality assurance standards in regulated industries
- Stay current with latest AI and GenAI testing methodologies and best practices

More Skills TypeScript, Playwright, Jest, Cypress, GitHub Actions, CI/CD pipelines, AWS, Grafana, Jira, GitHub, AI evaluation tools, AI/LLM testing frameworks, Automated UI/API/MCP testing, Machine learning validation, Generative AI evaluation, Agentic AI systems testing

Prepare for this role

Recommended resources to build the skills for this position. Sponsored.

Top 10 RAG Quick-Prep Questions

Zenaique A shorter RAG interview prep set for quick review.

Top 20 LLM System Design Questions (Quick)

Zenaique A focused LLM system design interview question set.

Top 50 LLM System Design Interview Questions

Zenaique

Curated LLM system design questions from Zenaique.

More LLM jobs

AI/LLM Safety Engineer

Propio Language Services

Leawood

Today

AI/ML Engineer 2

Day And Zimmermann

United States

Today

Software/AI developer

Rackner

United States

Today

Conversational AI Engineer

Bright Vision Technologies

United States

Today

Sr Lead Software Engineer - Agentic AI Solutions

JPMorgan Chase

Mumbai

Today

Senior Technical Consultant — AI Engineering & Agentic Salesforce Delivery

Salesforce

Bengaluru

Today

📌 Fullstack AI Quality Engineer – Applied & Agentic AI Systems (Hyderabad)
🏢 Top Gen AI Jobs
📍 Hyderabad