20 Nov
|
Spydra
|
Secunderabad
20 Nov
Spydra
Secunderabad
CPositions: 1 | Experience: 5–7 years
About the role You will design, develop and optimise distributed LLM inference systems that deliver high throughput, low latency and productive GPU utilisation.
Key responsibilities
- Deploy and optimise LLM serving using vLLM, SGLang, TensorRT-LLM and NVIDIA Dynamo.
- Implement appropriate tensor, pipeline and expert parallelism.
- Design prefill/decode disaggregation and optimise continuous batching, scheduling, speculative decoding and prefix caching.
- Develop cache-aware routing and load balancing across inference replicas.
- Integrate and evaluate LMCache, Mooncake and other distributed KV-cache technologies.
- Optimise long-context and high-concurrency workloads; diagnose compute, memory and scheduling bottlenecks.
Required technical skills Python, C++, PyTorch, vLLM, SGLang, NVIDIA Dynamo, Kubernetes, distributed systems, GPU memory management, model-serving APIs.
- Expected deliverables Scalable serving deployments, optimised configurations, measured performance gains and reproducible benchmarks.
📌 Senior Distributed Inference Engineer (Secunderabad)
🏢 Spydra
📍 Secunderabad