Job highlights

- Exposure to Microsoft Azure AI,OpenAI,LangChain,LangGraph,CrewAI,Semantic Kernel,or similar agent frameworks. .

Certifications such as: ISTQB Advanced Test Analyst / Test Manager,Certified Agile Tester,Azure AI Engineer Associate,Azure DevOps Engineer Expert.

- Experience with AI evaluation platforms and frameworks is preferred.

- Preferred Qualifications

Job Summary

About the Role Dailoqa is looking for an experienced Senior QA Engineer - Agentic AI Testing Evaluation to own the quality, testing, and evaluation strategy for our Broccoli Platform. This role will be responsible for establishing robust testing frameworks for AI agents, multi-agent workflows, orchestration layers, tools, and enterprise-grade AI applications. The ideal candidate will have deep expertise in test automation, quality engineering, AI/ML validation, security testing, and performance engineering, with a strong understanding of Large Language Models (LLMs), Agentic AI systems, and contemporary DevOps practices.

You will work closely with Capability Leads, AI Architects, Product Teams, and Platform Engineers to ensure that Broccoli delivers reliable, secure, scalable, and high-performing AI solutions.

Requirements

- 7-10 years of experience in Quality Assurance, Test Automation, or Quality Engineering.

- Strong experience with automation frameworks using Python, Java, or JavaScript/TypeScript.

- Hands-on experience with: Selenium, Playwright, Cypress, or Robot Framework.

- API testing tools such as Postman, Rest Assured, or Karate.

- CI/CD platforms including Azure DevOps, GitHub Actions, Jenkins, or GitLab.





- Experience with cloud platforms such as Azure, AWS, or Google Cloud.

- Agentic AI LLM Testing - Understanding of:

- Large Language Models (LLMs)

- Retrieval-Augmented Generation (RAG)

- AI Agents and Agent Orchestration Frameworks

- Prompt Engineering and Evaluation

- Experience with AI evaluation platforms and frameworks is preferred.

- Knowledge of hallucination detection, safety validation, guardrails, and model benchmarking.

- Performance Security Testing - Experience with performance testing tools such as JMeter, k6, or Locust.

- Knowledge of application security testing and threat modeling.

- Familiarity with AI security testing, adversarial testing, and governance controls.

- Soft Skills - Strong analytical and problem-solving skills. Excellent stakeholder management and communication abilities.

Experience working in cross-functional Agile teams. Ability to lead quality initiatives in fast-paced, innovation-driven environments.

Preferred Qualifications

- Experience testing AI-powered enterprise applications.

- Exposure to Microsoft Azure AI, OpenAI, LangChain, LangGraph, CrewAI, Semantic Kernel, or similar agent frameworks.

- Certifications such as: ISTQB Advanced Test Analyst/Test Manager, Certified Agile Tester, Azure AI Engineer Associate, Azure DevOps Engineer Expert.

- Experience working with BFSI, enterprise transformation, or regulated industry environments.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

📌 Senior QA Engineer - Agentic AI Testing (Noida)
🏢 Dailoqa
📍 Noida