Senior Distributed Inference Engineer (Hyderabad)

Senior Distributed Inference Engineer (Hyderabad)

19 Nov
|
Spydra
|
Hyderabad

19 Nov

Spydra

Hyderabad

CPositions: 1 |

Experience:
5–7 years

About the role
You will design, develop and optimise distributed LLM inference systems that deliver high throughput, low latency and productive GPU utilisation.

Key responsibilities
• Deploy and optimise LLM serving using vLLM, SGLang, TensorRT-LLM and NVIDIA Dynamo.
• Implement appropriate tensor, pipeline and expert parallelism.
• Design prefill/decode disaggregation and optimise continuous batching, scheduling, speculative decoding and prefix caching.
• Develop cache-aware routing and load balancing across inference replicas.
• Integrate and evaluate LMCache, Mooncake and other distributed KV-cache technologies.
• Optimise long-context and high-concurrency workloads; diagnose compute, memory and scheduling bottlenecks. Required technical skills Python, C++, PyTorch, vLLM, SGLang, NVIDIA Dynamo, Kubernetes, distributed systems, GPU memory management, model-serving APIs.
• Expected deliverables Scalable serving deployments, optimised configurations, measured performance gains and reproducible benchmarks.

📌 Senior Distributed Inference Engineer (Hyderabad)
🏢 Spydra
📍 Hyderabad

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: senior distributed inference engineer (hyderabad) / hyderabad

Subscribe to this job alert:

Get the latest job offers by email for: senior distributed inference engineer (hyderabad) / hyderabad