About Dusker AI

Dusker AI specializes in benchmarking and evaluating AI agents to help organizations understand real-world performance. Using expert-driven frameworks, we assess AI systems across reasoning, reliability, adaptability, and safety to ensure they are truly production-ready. From conversational AI to autonomous agentic systems, we build cutting-edge evaluation frameworks that enable organizations to develop trustworthy, high-performing AI solutions.

Role Overview

We are seeking an experienced Backend Engineer to design and scale the services that power our AI evaluation platform, from benchmark orchestration and sandboxed agent execution to trace ingestion and results APIs. You will own critical backend systems that run thousands of concurrent model and agent evaluations reliably, reproducibly, and at speed. You will work closely with evaluation scientists, front end engineers, and infrastructure colleagues to turn new scoring methodologies into dependable production services.

This role is ideal for someone passionate about distributed systems, API design, data-intensive pipelines, and the engineering craft behind rigorous AI measurement.

Key Responsibilities

- Design and build the distributed services that schedule, execute, and monitor large-scale agent evaluation runs.
- Architect data models and storage layers for benchmark definitions, agent traces, scores, and longitudinal result history.
- Build high-throughput job orchestration and queueing systems that sustain thousands of concurrent evaluation executions.
- Develop secure, sandboxed execution environments for agents that call external tools, APIs, and code interpreters.
- Define versioned internal and external APIs that expose benchmarks, run results, and evaluation artifacts to dashboards and integrations.
- Optimize query performance, caching strategy,



and compute cost across trace ingestion and analytics workloads.
- Collaborate with evaluation scientists to translate new scoring rubrics and metrics into production-grade services.
- Instrument the platform with structured logging, tracing, and alerting so failed runs are detected and diagnosed quickly.
- Document service architecture, data contracts, and operational runbooks for the wider engineering team.

Required Qualifications

- Bachelor's or Master's degree in Computer Science, Software Engineering, or a related technical field.
- 5+ years of qualified backend engineering experience building and operating production services at scale.
- Strong proficiency in Python, with working knowledge of a second backend language such as Go, Java, or TypeScript.
- Deep experience with API design and asynchronous service patterns using FastAPI, Django, or comparable frameworks.
- Hands-on expertise with PostgreSQL and at least one of Redis, Kafka, or Celery for caching, streaming, or task queues.
- Proven experience with Docker, Kubernetes, and CI/CD pipelines on AWS, Azure, or GCP.
- Practical familiarity with LLM APIs, tool calling, embeddings, vector databases, or retrieval-augmented generation systems.
- Clear written communication and the ability to work through ambiguous problems alongside research-minded colleagues.

Preferred Qualifications

- Experience building developer platforms, data pipelines, or machine learning experiment tracking systems.
- Familiarity with agent frameworks such as LangGraph, LangChain, CrewAI, or AutoGen.
- Background in observability tooling such as OpenTelemetry, Prometheus, or Grafana.
- Open-source contributions to backend, infrastructure, or LLM tooling projects.
- Exposure to multi-tenancy isolation, security review, or compliance requirements in enterprise software.

📌 Backend Engineer (Jaipur)
🏢 Dusker AI
📍 Jaipur