Job Title: GenAI Test Engineer (AI Security Testing)

Experience

5 - 10 Years

Job Summary

We are looking for a GenAI Test Engineer with strong experience in software testing and hands-on expertise in Generative AI, LLMs, and AI Security Testing. The ideal candidate will be responsible for validating AI-powered applications, ensuring the quality, reliability, security, and safety of AI model outputs. The role requires experience in testing LLM-based and RAG-based applications, automating test cases, validating prompt behavior, and identifying AI-specific risks such as hallucinations, bias, and prompt injection attacks.

Key Responsibilities

- Design and execute test strategies for Generative AI and Agentic AI applications.
- Validate LLM outputs for accuracy, consistency, hallucinations, bias, toxicity, and safety.
- Perform AI Security Testing, including prompt injection, jailbreak, adversarial testing, and OWASP LLM security validation.
- Test RAG (Retrieval-Augmented Generation) applications for retrieval quality, context relevance, and response accuracy.
- Create and execute manual and automated test cases for AI applications.
- Develop automation scripts using Python, Selenium, or Playwright.
- Perform API testing using Postman or similar tools.
- Execute regression testing for prompt updates, model upgrades, and AI workflow changes.
- Validate AI integrations, APIs, and end-to-end business workflows.
- Analyze test results, log defects, and work closely with developers to resolve issues.
- Integrate automated test suites into CI/CD pipelines.
- Manage AI test datasets and ensure comprehensive test coverage.

Required Skills




- 58 years of experience in Software Testing (Manual and Automation).
- Hands-on experience in GenAI/LLM Testing.
- Strong understanding of Large Language Models (LLMs) and Generative AI.
- Experience testing RAG-based applications.
- Knowledge of AI Security Testing, including Prompt Injection, Jailbreak Testing, Adversarial Testing, and OWASP LLM concepts.
- Robust programming skills in Python.
- Good knowledge of SQL.
- Experience with Selenium and/or Playwright.
- Strong experience in API Testing (Postman/REST APIs).
- Experience with CI/CD tools such as Jenkins, GitLab CI, or Azure DevOps.
- Understanding of AI model behavior, prompt engineering, and model variability.
- Experience with test data management and AI output validation.

Preferred Skills
- Experience with LangChain, LlamaIndex, or similar AI frameworks.
- Knowledge of Vector Databases (Pinecone, FAISS, ChromaDB, Milvus, etc.).
- Familiarity with cloud platforms such as Azure, AWS, or GCP.
- Experience with Responsible AI, AI Governance, and AI Risk Assessment.
- Exposure to performance and scalability testing for AI applications.

Qualifications
- Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.

Mandatory Skills
- GenAI Testing
- LLM Testing
- AI Security Testing
- RAG
- Python
- SQL
- Selenium or Playwright
- API Testing
- Test Automation
- CI/CD

This role is ideal for candidates with a strong QA background who have transitioned into GenAI and AI Security Testing, with practical experience validating LLM-powered applications and securing AI systems against emerging threats.

📌 Gen AI Test Engineer ( AI Security Testing ) (Bengaluru)
🏢 iKrux
📍 Bengaluru