At Delta, we are reimagining and rebuilding the financial system. Join our team to make a positive impact on the future of finance.

Mission Driven: Re-imagine and rebuild the future of finance.

Delta is bigger than all the Indian crypto exchanges combined.

The founding team is comprised of IIT and ISB graduates. and our tech co-founder is a serial entrepreneur who previously co-founded TinyOwl and Housing.com.

Funded by top crypto funds (Sino Global Capital, CoinFund, Gumi Cryptos) and crypto projects (Aave and Kyber Network).

AI Evaluation Engineer (SDET - AI)

Delta operates three AI products used by live traders: a customer support chatbot, an API Copilot that generates and executes trading scripts, and an MCP server. This role owns the quality of those products and is responsible for measuring it objectively.

Testing probabilistic systems differs fundamentally from testing deterministic ones. The core of this role is establishing what correct means for these systems, building the datasets and scoring that measure it, and operating the release gate that prevents a regression from reaching customers.

Most QA roles that mention AI mean using AI to do testing faster: generating test cases from a PRD, healing flaky selectors, exploring an app with an agent. The system under test is itself an AI, and the hard problem is deciding whether its output is correct when the same question produces a different answer every time and a wrong answer reads as convincingly as a right one. That means building evaluation datasets, defining what correct looks like, scoring against it, and defending a number that decides whether a release ships.

Build and maintain golden datasets for our AI products,



trace mined from production or generated, and verified against a documented source of truth

Design layered scoring for AI outputs: deterministic rules where behaviour can be asserted, and model-graded evaluation where it cannot

Convert findings into actionable engineering evidence and into permanent regression coverage

Define and report quality metrics for AI surfaces, and drive improvement against them

At least 1 year owning quality for AI products in a lead or primary-owner capacity. This includes chatbots and conversational assistants, code or content generation products, and agentic systems. You should have worked directly with prompts, tool and function schemas, and model behaviour, rather than testing around them

Demonstrated hands-on experience building or operating an evaluation harness for an AI system, whether in-house or using a framework such as DeepEval, RAGAS or Promptfoo. 4-6 years of QA / SDET experience

- Working proficiency in Python. Our evaluation platform, trace analysis and internal tooling are Python-based
- Ability to analyse traces and tool calls, using observability tooling such as Opik, Langfuse or LangSmith or an equivalent, to determine root cause rather than reporting the symptom
- Solid API testing experience. Bonus
- Experience with trading or exchange platforms, including familiarity with futures and options, crypto derivatives, margin, or settlement flows. Play a pivotal role in shaping the regulatory landscape for digital assets and Web3 in India

Be part of a mission-driven, fast-growing organisation at the forefront of financial innovation

Competitive compensation, leadership exposure, and significant growth opportunities

📌 Tech / AI Specialist (India)
🏢 Delta Exchange
📍 India