Skills

Robust in QA/Automation Testing concepts

Worked on AI related projects (Generative AI/RAG/LLM)

Location: Bangalore/Pune

Experience: 7 to 10 Years

Onboarding Date: 0-30 Days ONLY

Role overview

- Test Lead will drive QA strategy, governance, and delivery for an enterprise Agentic AI Platform supporting insurance workflows. This includes leading a team of testers, defining quality standards for LLM/agent behavior, overseeing safety and compliance, and ensuring

predictable, reliable operation of multi agent systems and RAG pipelines.
- Experience in leading AI/ML testing initiatives, managing test teams, ensuring quality and enterprise AI deployments. The Test Lead will be responsible for strategic planning, governance, stakeholder management, and delivery oversight, ensuring that AI solutions are

robust, ethical, and production ready.
- Experience in software testing, specializing in Artificial Intelligence (AI) systems and MLOps pipelines. The role involves validating AI/ML models, ensuring robustness of end-to-end machine learning workflows, and driving quality engineering practices across data, models, and deployment environments. The engineer will collaborate with data scientists, ML engineers, and DevOps teams to ensure trustworthy, scalable, and production-ready AI solutions.




- Strong understanding of both manual and automated testing methodologies, with the ability to design, develop, and execute test scripts.
- Responsible for the testing and delivery of parts of a product, in accordance with the customers requirements and interpersonal quality norms.
- Strong analytical and problem-solving skills, along with a proactive approach to identify and resolve issues, are essential.
- Responsible for ensuring that the testing process is efficient, effective, and aligned with project goals and timelines.

Requirement:
- 9-12 years of experience in software testing and quality engineering, with at least 3-4 years in AI testing.
- Proven leadership in managing test teams and delivery programs.
- Strong expertise in Python for automation and AI testing scripts.
- Knowledge of Gen AI frameworks (Langchain, Langgraph, CrewAI).).
- Experience in CI/CD pipelines and DevOps practices.
- Strong understanding of model evaluation metrics, drift detection, and monitoring.
- Exposure to AI ethics and compliance testing.
- Knowledge of cloud-native architectures (AWS, Azure, GCP).
- Familiarity with containerization & orchestration (Docker, Kubernetes).

📌 AI Testing Engineer (Bengaluru)
🏢 Black Turtle
📍 Bengaluru