About Gnani.ai

Gnani.ai builds voice-first AI for enterprises. Our products include an agentic AI platform, a voice API platform (speech APIs), and a conversation analytics platform. We build our own ASR, TTS, and small language models for 22+ Indian languages, serving BFSI, insurance, healthcare, and telecom customers at scale.

About the Role:

Our agentic AI platform lets enterprises build and run autonomous voice and chat agents. As Staff AI Engineer — Agentic AI, you will own the intelligence layer of this platform: how agents think, plan, retrieve knowledge, use tools, and work together. You will set the technical direction for LLM orchestration, RAG, and multi-agent systems, and lead a team of agentic AI engineers to ship it.

This is a hands-on leadership role. You will write code, review designs, and mentor engineers — not just manage.

What You Will Do:

LLM Orchestration

- Design and own the orchestration layer that routes requests across LLMs (hosted and self-hosted SLMs), with fallbacks, caching, and cost/latency controls.
- Build prompt management, structured output handling, and tool-calling pipelines that hold up in real-time voice conversations (strict latency budgets).

RAG Pipelines

- Own the end-to-end RAG stack: ingestion, chunking, embedding, retrieval, re-ranking, and grounding for enterprise knowledge bases.
- Improve answer accuracy and reduce hallucination for domain-heavy verticals (BFSI, insurance, healthcare), including code-mixed and multilingual content.
- Build freshness, versioning, and access control into retrieval so each tenant only sees its own data.

Multi-Agent Orchestration

- Design the multi-agent architecture: planner/worker patterns, agent hand-offs, shared memory, and inter-agent communication.
- Own agent memory design (contact, campaign, and agent-level memory)



and how agents learn from production feedback.

Tool Use & Enterprise Integrations

- Build the tool-calling and integration framework that lets agents take real actions: CRM updates, ticket creation, payment flows, and API calls into customer systems.
- Make tool execution safe and auditable: schemas, validation, retries, and human-in-the-loop approval where needed.

Evaluation & Observability

- Define evaluation gates: offline evals, golden test sets, persona simulators, and LLM-as-judge pipelines before changes ship.
- Build observability for every agent decision: traces, decision logs, and quality dashboards so failures can be found and fixed fast.

Guardrails, Safety & Compliance

- Design guardrails against prompt injection, hallucinated actions, and off-policy behavior, with deterministic fallbacks and state recovery.
- Ensure agent behavior meets enterprise compliance needs (data privacy, consent, and disclosure rules) in partnership with product and legal teams.

Cost & Performance Engineering

- Own inference cost and latency: model selection and routing, caching, batching, and KV-cache reuse, so agents stay fast and affordable at scale.

Team Leadership

- Lead and mentor a team of agentic AI engineers (roughly 4–8). Set direction, review designs and code, and raise the quality bar.
- Plan the agentic AI roadmap with product and platform teams. Break big goals into sprint-sized work.




- Hire and grow the team as the platform scales.

What You Bring:

Must have

- 8+ years in software or ML engineering, with 2+ years building LLM-based or agentic systems in production.
- Deep, hands-on experience with LLM orchestration frameworks and patterns (function calling, tool use, structured outputs, streaming) — and knowing when to skip the framework and build it yourself.
- Production RAG experience: vector databases, retrieval quality tuning, re-ranking, and eval-driven iteration.
- Experience designing multi-agent systems: task decomposition, agent coordination, memory, and failure handling.
- Strong Python; comfort with Go is a plus. Solid grasp of distributed systems (queues/messaging, Redis, Kubernetes).
- Track record of leading engineers as a tech lead or staff engineer: design reviews, mentorship, delivery ownership.

Nice to have

- Real-time or voice AI experience (latency-sensitive pipelines, streaming ASR/TTS integration).
- Fine-tuning or serving SLMs (vLLM, TensorRT-LLM, or similar).
- Experience with Indic languages or code-mixed text.
- Familiarity with enterprise compliance needs (data residency, DPDP, RBI guidelines).
- Hands on experience with Livekit and Pipecat frameworks

Why This Role

- Own a core layer of a fast-growing agentic platform used by large enterprises, end to end.
- Work on hard, real problems: agents that talk on live phone calls with sub-second latency budgets.
- Build on proprietary models (ASR, TTS, SLM) — not just API wrappers.
- Small, senior team. High trust, high ownership, direct access to leadership.

How We Work

Bengaluru-based, in-office collaboration. Sprint-based delivery with a explicit roadmap. Design docs and evals before big changes. We value engineers who ship, measure, and improve.

📌 Staff AI Engineer — Agentic AI (Bengaluru)
🏢 gnani.ai
📍 Bengaluru