19 Nov
|
Virtusa
|
Bengaluru
We are seeking a skilled AI/ML Engineer to lead GenAI Testing and LLM Testing for our next-generation Artificial Intelligence products. You will design automated validation frameworks to ensure the reliability, accuracy, and AI Security of autonomous workflows and conversational systems.
Key Responsibilities:
- Build and execute automated test suites for Generative Artificial Intelligence (Generative AI), Agentic AI, and Retrieval Augmented Generation (RAG) pipelines.
- Perform rigorous RAG Evaluation and Natural Language Processing (NLP) benchmarking to minimize hallucinations and ensure response quality.
- Develop and test integrations using the Model Context Protocol (MCP), validating context sharing across tools and models.
- Design and automate API testing workflows using Postman, Python, and Java.
- Collaborate with Machine Learning and Deep Learning teams to identify model vulnerabilities, enforce AI Security guardrails, and track defects using JIRA.
Required Skills & Qualifications:
- Solid programming experience in Python and Java for test automation and AIML application development.
- Hands-on experience with Generative AI, Agentic AI, Retrieval Augmented Generation, and Model Context Protocol (MCP).
- Proven expertise in LLM Testing, GenAI Testing, RAG Evaluation, and AI Security best practices.
- Solid understanding of core Machine Learning, Deep Learning, and Natural Language Processing concepts.
- Proficiency in API testing with Postman and agile project tracking with JIRA.
📌 AI/ML Quality & Automation Engineer (GenAI & Agentic Systems) (Bengaluru)
🏢 Virtusa
📍 Bengaluru