Role Overview

Zuppler is seeking an AI Quality Assurance & Evaluation Engineer to join our Product & Engineering team. This is not a traditional software QA position focused solely on standard UI clicks, manual test scripts, or basic automation frameworks.

In this role, you will be responsible for testing, benchmarking, and analyzing AI-driven feature architectures, conversational interfaces, and intelligent automation systems. You will systematically evaluate model performance, trace non-deterministic outputs, analyze system prompts, and design robust evaluation suites to ensure high accuracy, low latency, and seamless reliability across our ordering and platform integrations.

Key Responsibilities

- AI & LLM Feature Evaluation: Build and execute systematic benchmarking strategies to measure model accuracy, latency, context compliance, hallucination rates, and prompt performance across multi-step ordering workflows.
- Non-Deterministic Edge Case Analysis: Identify failure modes, unexpected user behaviors, and edge-case scenarios unique to conversational AI, voice ordering, and AI back-office automation tools.
- Evaluation Pipeline Automation: Design automated test suites and evaluation datasets (golden test sets) to validate model updates, regression risks, and prompt adjustments before production releases.
- Cross-Functional Collaboration: Partner directly with Product Management, Core Engineering,



and DevOps to translate feature requirements into rigorous test plans and evaluation action items.
- Performance & Quality Tracing: Analyze backend log traces, inspect prompt/response flows, and verify output data integrity across complex menu structures, options, and dependency logic.

Requirements & Qualifications

- Education: Bachelor’s degree in Computer Science, Software Engineering, Data Science, or a closely related technical field.
- Technical Background: Robust understanding of core computer science fundamentals (data structures, system design, algorithm analysis).
- AI/LLM Familiarity: Practical experience or strong conceptual understanding of testing AI/LLM applications, prompt engineering evaluation, or NLP pipelines.
- Scripting & Automation: Ability to write clean, reusable scripts for evaluation pipelines and test automation.
- Analytical Mindset: Exceptional problem-solving skills with a focus on statistical accuracy, data integrity, and root-cause failure analysis.

Nice-to-Haves

- Experience testing voice-driven AI tools, conversational chatbots, or automated ordering workflows in restaurant tech, SaaS, or e-commerce environments.
- Exposure to modern AI evaluation frameworks.
- Familiarity with containerized staging environments and standard CI/CD pipelines.

📌 AI Quality Assurance & Evaluation Engineer (New Delhi)
🏢 Zuppler
📍 New Delhi