Role OverviewZuppler is seeking an AI Quality Assurance & Evaluation Engineer to join our Product & Engineering team. This is not a traditional software QA position focused solely on standard UI clicks, manual test scripts, or basic automation frameworks. In this role, you will be responsible for testing, benchmarking, and analyzing AI-driven feature architectures, conversational interfaces, and intelligent automation systems. You will systematically evaluate model performance, trace non-deterministic outputs, analyze system prompts, and design robust evaluation suites to ensure high accuracy, low latency, and seamless reliability across our ordering and platform integrations.

Key Responsibilities
• AI & LLM Feature Evaluation: Build and execute systematic benchmarking strategies to measure model accuracy, latency, context compliance, hallucination rates, and prompt performance across multi-step ordering workflows.
• Non-Deterministic Edge Case Analysis: Identify failure modes, unexpected user behaviors, and edge-case scenarios unique to conversational AI, voice ordering, and AI back-office automation tools.
• Evaluation Pipeline Automation: Design automated test suites and evaluation datasets (golden test sets) to validate model updates, regression risks, and prompt adjustments before production releases.
• Cross-Functional Collaboration: Partner directly with Product Management, Core Engineering,



and DevOps to translate feature requirements into rigorous test plans and evaluation action items.
• Performance & Quality Tracing: Analyze backend log traces, inspect prompt/response flows, and verify output data integrity across complex menu structures, options, and dependency logic. Requirements & Qualifications
•

Education:
Bachelor’s degree in Computer Science, Software Engineering, Data Science, or a closely related technical field.
• Technical Background: Strong understanding of core computer science fundamentals (data structures, system design, algorithm analysis).
• AI/LLM Familiarity: Practical experience or strong conceptual understanding of testing AI/LLM applications, prompt engineering evaluation, or NLP pipelines.
• Scripting & Automation: Ability to write clean, reusable scripts for evaluation pipelines and test automation.
• Analytical Mindset: Exceptional problem-solving skills with a focus on statistical accuracy, data integrity, and root-cause failure analysis. Nice-to-Haves
• Experience testing voice-driven AI tools, conversational chatbots, or automated ordering workflows in restaurant tech, SaaS, or e-commerce environments.
• Exposure to up-to-date AI evaluation frameworks.
• Familiarity with containerized staging environments and standard CI/CD pipelines.

📌 AI Quality Assurance (New Delhi)
🏢 Zuppler
📍 New Delhi