Location: Noida or Bengaluru

About the Role

Serving models efficiently is its own discipline, distinct from training them. As an Inference Runtime Engineer, you'll build the serving infrastructure that keeps GPU utilization high and cost per token low across everything we run, from rollout generation to production model serving.

What You'll Do
Build and optimize model-serving infrastructure: KV-cache management, continuous batching, and speculative decoding
Implement quantization (FP8/INT4) and intelligent routing to reduce serving cost per token
Support low-latency rollout serving for both internal model training and customer-facing workloads
Diagnose and resolve GPU utilization bottlenecks in production serving
Work with product engineering to expose serving capabilities through customer-facing APIs

What You'll Bring




5+ years of experience in systems or ML infrastructure engineering
Robust experience with model-serving systems (vLLM, SGLang, TensorRT-LLM, or comparable)
Hands-on CUDA or Triton experience
Solid understanding of quantization and batching strategies for inference
Proficiency in C++ and Python

Education

B.Tech in Computer Science or a related field; M.Tech a plus for senior scope.

Nice to Have
Experience at an inference-focused AI infrastructure company or a hyperscaler's model-serving team
Exposure to speculative decoding or other latency-reduction techniques in production

📌 Performance Engineer Inference Bengaluru
🏢 HCLTech
📍 Bengaluru