NVIDIA has continuously reinvented itself for over two decades. The invention of the GPU in 1999 fueled the growth of PC gaming, redefined up-to-date computer graphics, and revolutionized parallel computing. More recently, GPU deep learning has ignited modern AI, positioning NVIDIA as a leading AI computing company.
There is an increasing focus on delivering AI models locally, closer to the source of data. This reduces latency, improves real-time processing, and addresses privacy concerns by minimizing data transfer to centralized servers. As technology advances, client-side AI (local execution) will play a key role in shaping digital experiences.
The LocalAI team is seeking a Senior Systems Software Engineer to build efficient on-device AI software for RTX and DGX-class systems. This role focuses on high-performance local inference, low latency, efficient memory use, infrastructure and practical deployment on resource-constrained platforms.
What you'll be doing :
- Partnering with NVIDIA's software, research, architecture, and product teams to align strategies and technical needs,
encouraging the ecosystem of AI on RTX and DGX PCs.
- Building and optimizing local AI inference stack for RTX, RTX Pro and DGX GPUs, focusing on performance, stability, and scalability across various hardware architectures.
- Architecture and development of modern inference runtimes and execution stacks, covering frameworks like Llama.cpp, vLLM, PyTorch, WinML, DXCGC, and TensorRT-RTX across LLMs, vision-language, TTS, ASR, and diffusion AI workloads.
- Perform end-to-end optimization of AI models, data pipelines, and inference runtimes to enhance performance across current and next-generation GPU architectures. Apply model optimization techniques such as quantization, pruning, sparsity, and distillation to enable efficient deployment of large models on local and edge devices.
- Perform system-level debugging, performance optimization, and performance–accuracy trade-off analysis; develop
📌 Senior System Software Engineer (Pune)
🏢 NVIDIA
📍 Pune