Job Summary

Dive in and do the best work of your career at DigitalOcean. Journey alongside a strong community of top talent who are relentless in their drive to build the simplest scalable cloud. If you have a growth mindset, naturally like to think big and bold, and are energized by the fast-paced environment of a true industry disruptor, you ll find your place here. We value winning together while learning, having fun, and making a profound difference for the dreamers and builders in the world.

What You'll Do

- Technical Leadership: Act as an AI Inference lead on the FDE team, driving the end-to-end design, development, and delivery of critical AI workloads leveraging large generative AI models.
- Design & Scale Distributed AI Inference Systems: Architect and deploy production-grade, multi-tenant LLM inference engines using Kubernetes-native frameworks like llm-d, NVIDIA Dynamo, Ray Serve, vLLM, and SGLang.
- Lead Forward Deployed Engagement: Embed alongside external tech leads to debug latency spikes, profile GPU memory utilization, and refactor inference code for high-concurrency production workloads. You are the bridge between our customers and our internal AI infrastructure teams.
- Optimize at Cluster Scale: Solve the distributed-systems problems unique to LLM serving. You ll implement strategies for prefill/decode disaggregation, KV-cache-aware routing, tiered prefix caching, and wide expert parallelism for MoE models.
- Drive Hardware Efficiency: Guide customers on compute efficiency utilizing techniques like tensor/data parallelism, continuous batching, and quantization (FP8/FP4) to inflate tokens-per-second per dollar.
- Build Internal Tooling & Upstream Value:



Translate customer edge cases into reusable internal blueprints and contribute performance fixes directly back to open-source inference ecosystems (vLLM, llm-d) on behalf of DigitalOcean.
- Travel & Collaboration Requirements: Ability to travel up to 30% for customer engagements, strategic workshops, conferences, and internal collaboration. Ability to consistently overlap with North American business hours, including availability until at least noon Eastern Time, to collaborate effectively with customers, Product, Engineering, and go-to-market teams.

What You'll Add to DigitalOcean

- Deep Distributed Inference Fluency: 6+ years in AI/ML systems, with a deep understanding of why cluster-scale serving is hard (e.g., partitioning KV-cache across workers, fast cross-pod KV transfer, and inference-aware load balancing).
- Framework Mastery: Hands-on experience with vLLM, llm-d, SGLang, TensorRT-LLM, or Modular MAX, including a solid grasp of internals like continuous batching and paged attention.
- Code & Architecture Proficiency: Expert-level proficiency in Python or GoLang, familiarity with gRPC, and experience running critical services on Kubernetes in high-scale environments.
- Customer-Facing Engineering Mindset:



A clear communicator with a growth mindset who enjoys pairing with external engineering teams. You can seamlessly translate business latency SLAs into deep technical infrastructure solutions.
- The DO "Shark" Mentality: You think big, bold, and scrappy. You have a bias for action and a powerful sense of ownership over the customer experience.

Preferred Qualifications

AI Inference & Forward Deployed Engineering Experience: 6+ years of experience working in Forward Deployed Engineering, AI Inference architect, Technical Consulting roles supporting production AI systems.

- Customer Empathy & Technical Leadership: Ability to translate complex business tasks into AI engineering solutions and collaborate directly with client teams (CTOs, AI Leads).
- Builder Mentality: Preference for delivering production-ready code, low-latency container images, and deployment blueprints over slide decks.
- Agility: Comfortable navigating rapid-moving environments and tuning model workloads for diverse accelerator architectures.
- Vendor & Strategic Partnership Collaboration: Experience collaborating with GPU vendors, infrastructure providers, model vendors, or ecosystem partners on benchmarking, optimization, technical validation, or launch readiness initiatives.

This job is located in Bengaluru, India

Key Skills

- Python
- GoLang
- Kubernetes
- gRPC
- vLLM
- llm-d
- SGLang
- TensorRT-LLM
- Ray Serve

Disclaimer : This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

📌 Senior Forward Deployed Engineer I (AI Inference) (Bengaluru)
🏢 DigitalOcean
📍 Bengaluru