This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Inference Engineer based in India.
This is an opportunity to become the first dedicated engineer responsible for building and owning an inference platform from the ground up.
You’ll work closely with the CTO to turn large language models into reliable, production-grade query-to-response systems running at scale.
The role combines hands-on infrastructure engineering with performance optimization across GPU-based environments.
You’ll work with up-to-date serving technologies such as vLLM, SGLang, and TensorRT-LLM while solving real-world challenges around latency, cost, and throughput.
As the platform matures, you’ll take increasing ownership of its technical direction and collaborate with Product on the future inference roadmap.
You’ll join a remote-first, open-source-oriented startup where engineering ownership, speed, and measurable customer impact are highly valued.




This role is ideal for a senior engineer who wants significant autonomy and the opportunity to define how production inference infrastructure is built and scaled.
n

Accountabilities
Build and deploy production-grade LLM inference systems across one or multiple GPU machines, owning the complete pipeline from customer query to served response.
Design, implement, and operate model-serving infrastructure using technologies such as vLLM, SGLang, and TensorRT-LLM.
Optimize inference workloads for scale, balancing latency, throughput, reliability, and infrastructure costs.
Apply techniques such as quantization, batching, caching, and intelligent request routing to improve inference performance.
Develop robust production infrastructure using Python or Golang, with an emphasis on maintainable, scalable engineering rather than configuration-only work.
Establish the initial inference platform in close collaboration with the CTO and take ownership of its evolution as the organizati

📌 Senior Inference Engineer (India)
🏢 Jobgether
📍 India