About the role
You will design, develop and optimise distributed LLM inference systems that deliver high throughput, low latency and productive GPU utilisation.
Key responsibilities
• Deploy and optimise LLM serving using vLLM, SGLang, TensorRT-LLM and NVIDIA Dynamo.
• Implement appropriate tensor, pipeline and expert parallelism.
• Design prefill/decode disaggregation and optimise continuous batching, scheduling, speculative decoding and prefix caching.
• Develop cache-aware routing and load balancing across inference replicas.
• Integrate and evaluate LMCache, Mooncake and other distributed KV-cache technologies.
• Optimise long-context and high-concurrency workloads; diagnose compute, memory and scheduling bottlenecks. Required technical skills Python, C++, PyTorch, vLLM, SGLang, NVIDIA Dynamo, Kubernetes, distributed systems, GPU memory management, model-serving APIs.
• Expected deliverables Scalable serving deployments, optimised configurations, measured performance gains and reproducible benchmarks.