Title: Gen AI Tester

Location: Bangalore (Mandatory Work from Office 4 Days a week)

Mode of Interview: F2F

Experience Required : AI side 3+ Years, in testing side 8+ and 10+ years required.

Notice: Immediate Joiners only.

Role Overview:

We are seeking an AI-Native Quality Tester to redefine how we ensure the excellence of our digital experiences. In this role, you will go beyond traditional testing by developing AI Models to test AI agents. Leveraging artificial intelligence to predict defects, automate complex analysis, and optimize the quality of agents. You will be at the forefront of our "AI-first" mission

Key responsibilities:

Functional & Model Testing
- Design and execute test cases for AI/ML models, including classification, NLP, recommendation, and generative AI systems
- Validate model outputs for accuracy, consistency, and expected behavior across diverse inputs
- Perform regression testing after model updates, retraining cycles, or data changes
- Model Evaluation: Testing LLM outputs for hallucinations, bias, and correctness (using tools like LangSmith/Ragas).




- AI-Driven Automation: Building self-healing test automation frameworks using Python or JavaScript.
- Data Validation: Ensuring training/input data is clean, balanced, and free from bias.

Prompt & LLM Testing
- You possess hands-on experience with prompt engineering to craft accurate queries that improve the speed and accuracy of test generation and data synthesis. Excellent exposure to AI Tools like Claude, Gemini, UiPath and others)
- Define LLM evaluation strategies
- Develop and maintain prompt test libraries for large language model (LLM)-based features
- Evaluate model responses for relevance, tone, factual accuracy, and adherence to business rules
- Identify hallucinations, response drift, and edge-case failures in generative outputs
- Design and execute test cases to evaluate the accuracy, performance, and reliability of AI models
- Adversarial/Red Teaming: Intentionally breaking m

📌 Gen AI Tester (Bengaluru)
🏢 Cerebra
📍 Bengaluru