Hiring for FlexAI — via Flexiple, the recruiting partner on this role.

About the Company

FlexAI is building a high-performance, cloud-agnostic AI compute platform for training and inference. Its Token Factory serves 20+ open models through one OpenAI-compatible API, from serverless to dedicated endpoints and private cloud, running across NVIDIA, AMD and emerging AI accelerators. The company is led by the team that deployed the Aurora supercomputer and managed 50,000+ GPUs at Intel, is headquartered in Paris, and is growing its Bengaluru engineering centre as inference demand scales.

About the Role

As an AI Inference Engineer you'll design, build and optimise the serving stack that runs large language and multimodal models in production on heterogeneous hardware. It's a hands-on role at the intersection of model architecture, hardware and production serving: you'll own critical parts of an inference runtime built on vLLM, SGLang and PyTorch, help set direction on performance, and work with engineering, research and product to deliver quick, reliable and cost-efficient tokens to customers.

What you'll do

- Own the serving stack for LLMs and multimodal models: engine configuration, tensor/expert/data parallelism and deployment across NVIDIA and AMD GPUs.
- Bring new open-weight models to production quickly and correctly: chat templates, tool-calling and reasoning parsers, structured outputs, quantized checkpoints.
- Profile and optimise throughput, time to first token, inter-token latency and cost per token, from kernels to the request path.
- Apply and evaluate FP8/FP4 quantization, speculative decoding,



KV-cache reuse and offloading, prefix caching and prefill/decode disaggregation.
- Maintain and contribute patches to vLLM and SGLang, and upstream fixes where it makes sense.
- Run inference on Kubernetes across clusters and clouds with autoscaling, zero-downtime rollouts and graceful failure handling.
- Build observability, benchmarking, load-testing, evaluation and model-release tooling that catches silent quality regressions before customers do.

What you bring

- 2+ years building performance-critical systems software in Python and C++.
- Hands-on experience serving LLMs in production with vLLM, SGLang, TensorRT-LLM or similar engines.
- A deep understanding of transformer inference: attention and the KV cache, batching and scheduling, and when a workload is memory-bandwidth bound versus compute bound.
- Experience profiling GPU workloads (Nsight, rocprof, PyTorch profiler) and turning findings into measured improvements.
- Familiarity with distributed inference: tensor and expert parallelism, NCCL/RCCL, high-speed interconnects.
- Comfort with Kubernetes, containers and running production services.
- A habit of measuring before and after every change, and of verifying correctness, not only speed.

Nice to have

- Contributions to open-source inference engines or GPU kernels (CUDA, HIP, Triton).
- Experience with AMD ROCm or non-GPU AI accelerators.
- Experience with KV-cache systems (e.g. LMCache, Mooncake), speculative decoding or disaggregated serving.
- Experience with model quantization and accuracy evaluation.

Originally posted on: 5 Nov 2026

📌 AI Inference Engineer (India)
🏢 Flexiple
📍 India