Job Role : Test Engineer - AI

Experience : 6+ Years

Location : Bangalore

Job Summary :

We are looking for an experienced Test Engineer AI to design, develop, and execute comprehensive testing strategies for AI/ML-based applications and platforms. The ideal candidate should have strong expertise in software testing, test automation, AI/ML testing, API testing, and performance testing, with the ability to validate AI models and AI-powered applications for quality, reliability, accuracy, and scalability.

Key Responsibilities :

- Design and implement comprehensive test strategies, test plans, test scenarios, and test cases for AI/ML applications.
- Perform functional, regression, integration, system, API, and end-to-end testing.
- Develop and maintain automated test frameworks using Python and relevant automation tools.
- Test AI/ML models, LLM applications, RAG pipelines, chatbots, and AI-powered APIs.
- Validate model outputs for accuracy, consistency, relevance, bias, hallucinations, and robustness.
- Design testing approaches for LLM prompts, embeddings, vector databases, RAG workflows, and AI agents.
- Perform API testing using tools such as Postman, REST Assured, or PyTest.
- Integrate automated tests into CI/CD pipelines.
- Conduct performance, scalability, and reliability testing of AI applications and services.
- Identify defects, perform root-cause analysis, and work closely with developers, data scientists, and ML engineers to resolve issues.
- Develop test automation utilities, frameworks, and reusable components.
- Monitor test results, analyze failures, and provide detailed quality reports.
- Ensure testing aligns with security, reliability, data quality, and responsible AI requirements.

Required Skills :





- 6+ years of experience in Software Testing / Test Automation.
- Strong programming experience in Python.
- Hands-on experience with AI/ML testing or testing AI-powered applications.
- Strong knowledge of Selenium, Playwright, PyTest, or equivalent automation frameworks.
- Experience with API testing and RESTful services.
- Knowledge of LLMs, Generative AI, RAG, Prompt Engineering, Embeddings, and Vector Databases.
- Experience testing AI model responses, hallucinations, accuracy, relevance, and consistency.
- Strong understanding of SQL and database testing.
- Experience with Git, Docker, and CI/CD tools such as Jenkins, GitLab CI, or GitHub Actions.
- Familiarity with cloud platforms such as AWS, Azure, or GCP.
- Strong knowledge of SDLC, STLC, Agile/Scrum, defect lifecycle, and test methodologies.

Good to Have :

- Experience with LangChain, LlamaIndex, Hugging Face, OpenAI APIs, or similar AI frameworks.
- Knowledge of AI evaluation frameworks and metrics such as BLEU, ROUGE, Recall@K, Precision@K, MRR, and faithfulness.
- Experience with MLflow, Kubernetes, or cloud-based AI/ML platforms.
- Knowledge of AI security, prompt injection, data privacy, and responsible AI testing.
- Experience with performance testing tools such as JMeter or Locust.

Education :

- Bachelors/Masters degree in Computer Science, Information Technology, Engineering, or a related field.

Key Competencies :

- Strong analytical and problem-solving skills.
- Excellent debugging and root-cause analysis abilities.
- Strong attention to detail.
- Positive communication and collaboration skills.
- Ability to work independently in a fast-paced Agile environment.
- Strong understanding of quality engineering principles.

📌 AI Test Engineer (India)
🏢 FUNIC TECH PRIVATE
📍 India