About the Team & Mission The ML Platform team maximizes the speed, throughput, and hardware efficiency of our production Large Language Model (LLM) inference stack. As models scale and real-time demand grows, post-training optimization and aggressive server-level utilization are our primary levers for delivering low-latency AI at sustainable infrastructure economics.

We operate at the intersection of LLM serving engines, post-training compression, and bare-metal accelerator architecture. Our mission is to squeeze maximum compute and memory bandwidth out of every server node—turning heavy foundation models into ultra-quick, cost-efficient endpoints.

Role Overview

We are hiring a deeply technical Senior / Staff Software Engineer (Individual Contributor) with 8+ years of experience to partner directly with the Leadership in architecting and scaling our LLM inference platform.

In this high-leverage IC role, you will serve as our primary technical anchor for inference performance. You will own end-to-end serving architecture, drive post-training optimization pipelines, eliminate hardware bottlenecks, and guide engineers through complex systems challenges.

Key Responsibilities

- Post-Training Optimization: Architect and productionize model compression and acceleration pipelines—including quantization (FP8/INT8/INT4), distillation, and speculative decoding—to slash latency (TTFT/ITL) while preserving model quality.
- Hardware & Server Maximization: Drive bare-metal utilization across multi-GPU and multi-node topologies by optimizing memory bandwidth, tensor core occupancy, interconnects, and parallelism strategies (tensor, pipeline, expert, and disaggregated serving).




- Inference Engine & Kernel Engineering: Extend modern inference frameworks (such as vLLM or SGLang) and build custom fused kernels and batching schedulers to eliminate compute, memory, and CPU host bottlenecks.
- Profiling & Automated Diagnostics: Build deep telemetry, roofline modeling, and automated bottleneck-detection workflows to pinpoint hardware stalls and prevent performance regressions in production.
- Technical Leadership: Partner with Leadership to shape our multi-quarter inference roadmap, author high-clarity RFCs and influence hardware capacity decisions.
- Mentor Junior Engineers: Level up junior and mid-level engineers through code reviews, pair programming, and hands-on systems coaching.

Minimum Qualifications

- Education: Bachelor’s or Master’s degree in Computer Science, Computer Engineering, Electrical Engineering, or equivalent practical systems experience.
- Experience: 8+ years of hands-on software engineering experience in high-performance computing (HPC), ML inference systems, GPU programming, or low-latency distributed infrastructure.
- LLM Inference Mastery: Proven track record optimizing large-scale LLM serving in production, with a deep understanding of transformer memory footprints, KV-cache mechanics, continuous batching, and latency/throughput trade-offs.




- Hardware & Systems Depth: Strong grasp of accelerator microarchitecture (compute vs. memory-bound workloads, roofline analysis, memory hierarchies) and multi-GPU server interconnects.
- Programming Proficiency: Expert-level coding skills in C++ and Python, with experience building low-overhead, production-critical systems software.

Preferred Qualifications

- GPU & Compiler Internals: Practical experience writing or profiling custom accelerator kernels and working close to ML compilers or runtime execution graphs.
- Production Compression & Decoding: Track record deploying quantized models or speculative decoding techniques in high-throughput production environments.
- Open-Source Inference Familiarity: Experience working inside or contributing to modern open-source LLM serving engines and attention libraries.
- Complex Workload Scaling: Background optimizing Mixture-of-Experts (MoE) routing overhead or ultra-long-context inference workloads.
- Automated Performance Tooling: Experience building automated benchmarking harnesses, hardware utilization models, or AI-assisted trace debugging tools.

Why Flipkart?

Flipkart is India’s leading e-commerce platform, where innovation meets impact at scale. As part of the AI Solutions team, you will shape the foundational AI infrastructure powering experiences for 500M+ users, millions of sellers, and complex supply chains. Flipkart is part of the Walmart-owned Flipkart Group, which includes Flipkart Wholesale, Flipkart Health+, Cleartrip, Shopsy, Flipkart Travels, and Myntra. Join us to build the platforms that make AI at Flipkart possible.

📌 Senior AI Engineer (Bengaluru)
🏢 Flipkart
📍 Bengaluru