Senior AI / RAG Engineer — Applied LLM & Retrieval

This is a deeply hands-on role for an experienced engineer who wants to spend their time building. A big part of the job is growing our fleet of AI sub-agents — designing new specialized agents and wiring them into complete, end-to-end multi-agent agentic workflows and architecture. You'll design, implement, and harden our RAG, agent, and LLM features end-to-end — the retrieval pipelines, the agent orchestration, the model-gateway routing and fallback logic, the AI automation pipelines, and the evaluations that keep quality high — all inside a GDPR-first, EU-only data boundary. Your work ships to production and you own it through to it working reliably.

What you'll do

- Grow our fleet of AI sub-agents — design and build more and more specialized agents, and wire each into a complete multi-agent, agentic AI-driven workflow and architecture, from single-purpose agents up to orchestrated end-to-end flows.
- Build and improve RAG pipelines — chunking, embeddings, vector storage and retrieval, re-ranking, grounding and citation — over proprietary data such as work orders, rate cards, and uploaded documents.
- Build LLM features on cloud-hosted large language models — classification, extraction, summarisation, structured/JSON output, prioritisation, and multimodal (text + image) reasoning — from prototype through to production.
- Extend the internal Model Gateway — task/tier model routing, retries, fallback, cost estimation, and per-tenant usage limits.
- Build adversarial validation agents — AI agents that critique and stress-test model outputs, plans, and designs over multiple rounds to catch errors and edge cases before they ship.
- Design and build AI automation pipelines — orchestrated, repeatable AI workflows,



both in the product and across our engineering process.
- Automate the SDLC with AI — bring AI into how we plan, generate, review, test, and ship code, and build the tooling that makes it repeatable.
- Integrate and extend an automated testing framework — including AI-assisted test generation and self-checking pipelines for our AI features.
- Build the evaluations — datasets and harnesses that measure accuracy, faithfulness/hallucination, latency, and cost, and catch regressions in non-deterministic systems before release.
- Do the prompt engineering — schema-validated outputs, tool/function calling, and the tuning needed to reduce hallucination and hit quality targets.
- Enforce data-protection by design — tenant-scoped retrieval, PII handling, and erasure/cascade in the retrieval layer; keep every data flow inside the EU/EEA.
- Ship to production on AWS (Python, ECS) with proper observability, and own your features through to reliable operation.

Required skills & experience

- Strong experience in software engineering, with deep, production-grade Python.
- Substantial hands-on experience building and shipping LLM-powered systems to production — you've taken more than one from idea to reliable, scaled feature and stayed close to the code.
- Designing and composing multi-agent systems — building specialized sub-agents and wiring them into complete,



end-to-end agentic workflows and architecture.
- Deep RAG expertise — embeddings, vector stores (e.g. pgvector, OpenSearch, Pinecone, FAISS), semantic search, chunking strategies, re-ranking, grounding — and the judgement to know what actually moves quality.
- Embeddings at scale and multilingual retrieval.
- Advanced prompt engineering and structured output / tool use / function calling with schema-validated (JSON) responses; solid instincts for reducing hallucination.
- Deep experience with cloud-hosted commercial LLM APIs / a managed LLM platform.
- Agentic / multi-agent frameworks (LangGraph, LangChain, or similar) — orchestration and state machines — including building adversarial / validation agents or LLM-as-judge setups.
- AI-driven SDLC automation and experience building AI automation pipelines (CI/CD or workflow orchestration).
- Integrating automated testing frameworks and AI-assisted test automation.
- Proven ability to build evaluations for AI systems — datasets, metrics (accuracy, faithfulness, latency, cost), and regression testing.
- Real command of cost/latency trade-offs — model tiering, routing, fallback, and caching — at production scale.
- Strong AWS background — building, deploying, and operating services in production; IAM and region-aware design.
- Streaming, latency optimisation, and prompt caching.
- Vector-DB operations, observability (Datadog / CloudWatch), and IaC.
- GDPR / data-residency / responsible-AI experience — no-training commitments, tenant isolation, DPAs, PII minimisation.
- Experience with AI systems running at production scale.
- Excellent testing discipline, code quality, and engineering judgement; comfortable owning a feature end-to-end.

📌 Senior AI / RAG Engineer — Applied LLM & Retrieval (Hyderabad)
🏢 Bnyn Softtech
📍 Hyderabad