At RoleChain, we don’t just serve LLM endpoints—we engineer high-throughput, sub-millisecond, low-latency inference fabrics that power deterministic multi-agent execution at enterprise scale. About RoleChainModern AI agents fail when latency spikes, GPU memory bottlenecks, or batching collapses under dynamic loads. At RoleChain, we are building the resilient infrastructure backbone for autonomous enterprise AI workflows. We design deterministic, self-healing agent architectures that process complex reasoning chains in real time, without sacrificing speed, throughput, or operational budget. To achieve this, we treat Inference as an Architectural Discipline. We optimize from the bare metal up—from CUDA kernels and custom FlashAttention pipelines to distributed vLLM clusters, dynamic KV-cache compression, and intelligent model partitioning.

About the Role
We are looking for a world-class AI Inference Architect to lead the architecture, optimization, and scaling of RoleChain’s real-time AI serving engines. In this role, you will bridge hardware-level GPU compute, model acceleration frameworks, and distributed inference topologies. You will design ultra-low-latency, high-concurrency systems capable of serving frontier open-weights and proprietary models (LLMs, Vision-Language Models, and Speculative Decoding agents) under rigorous enterprise SLA guarantees. You’re Excited About This Prospect Because You Will…
• Architect Serving Infrastructure: Design, benchmark, and deploy enterprise-grade LLM inference platforms using frameworks like vLLM, TensorRT-LLM, TGI (Text Generation Inference), and SGLang.
• Optimize GPU & Memory Efficiency: Master KV-cache management (PagedAttention,



Chunked Prefill), quantization schemes (AWQ, FP8, INT4, GPTQ), dynamic batching, and speculative decoding to squeeze maximum throughput out of H100/A100/L40S clusters.
• Drive Distributed Compute Topologies: Implement Tensor Parallelism (TP), Pipeline Parallelism (PP), and Multi-Node GPU cluster configurations to serve massive multi-billion parameter models with sub-second time-to-first-token (TTFT) and high inter-token generation speed.
• Reduce Latency & Compute Costs: Formulate and execute hardware-aware software optimizations (CUDA, Triton kernels, Operator Fusion) to drastically lower the total cost per million tokens while maintaining SLA compliance.
• Build Telemetry & Observability Systems: Implement end-to-end inference monitoring for GPU utilization, memory fragmentation, latency distributions (p95/p99), prefix-caching hit rates, and model degradation guardrails.
• Partner Across Engineering: Work closely with our Agentic Systems and Applied AI teams to co-design models and prompt execution pipelines optimized for serving efficiency. We’re Looking for Someone Who Has…
• Deep Systems & Infrastructure Background: 15+ years of software/systems engineering experience, with 3+ years specifically architecting high-throughput production ML/AI inference pipelines at scale.




• Model Acceleration Expertise: Advanced hands-on experience with TensorRT-LLM, vLLM, ONNX Runtime, CUDA, Triton, or TVM. You know how models execute down to the tensor core and memory bandwidth levels.
• Mastery of Quantization & Distillation: Practical experience implementing FP8/FP16/INT4 quantization, model pruning, and speculative sampling techniques without sacrificing task accuracy.
• Cloud & Distributed Systems Execution: Proven experience with Kubernetes, Ray, Triton Inference Server, NVIDIA NCCL, and cloud-native orchestration on GCP/AWS/Azure.
• Systems Thinking & Pragmatism: You understand the trade-offs between throughput vs. latency, memory footprint vs. accuracy, and cold-start overhead vs. warm-cache cost. Preferred Qualifications
• Experience writing custom CUDA or OpenAI Triton kernels for custom attention mechanisms or speculative decoding.
• Contributions to open-source inference engines (vLLM, TensorRT-LLM, SGLang, Ollama, etc.).
• Deep understanding of hardware architecture across NVIDIA Hopper/Blackwell, AMD Instinct (ROCm), or custom AI accelerators (TPUs/AWS Inferentia). Why Join RoleChain?
• High-Impact Architecture: Direct ownership over the inference performance governing thousands of real-time multi-agent workflows.
• Frontier Technical Challenges: Solve real bare-metal and distributed systems problems at the bleeding edge of AI serving technology.
• Engineering-First Culture: Work alongside elite systems engineers who value clean architecture, benchmark-driven development, and deep technical rigor over hype.

📌 AI Inference Architect (India)
🏢 RoleChain
📍 India