Overview
- Join our team AIInfra, where we are building the AI data-plane that powers all LLM inferencing workloads across Microsoft and Azure customers—from cutting-edge startups to Fortune 500 enterprises. Our converged AI fabric delivers inference capabilities for all LLMs in Microsoft catalog, including OpenAI, Anthropic, Mistral, Cohere, Llama, and more.

Job Summary

As a Principal Software Manager - Ai data-plane, you will shape the future of one of the largest and fastest-growing services in Azure, foundational to Microsoft’s AI strategy. Our mission is to serve models at scale—reliably, efficiently, and with ultra-low latency—enabling a rich set of AI-powered product experiences.
- This is a rapidly evolving space with immense opportunities to learn, innovate, and drive industry-wide impact You will lead a 50 person team to deliver products at scale with the most cutting edge technology.
- Build and scale the core serving systems and smart request routing and distribution for all LLMs (OpenAI, Anthropic, Mistral, Grok, DeepSeek and many others).
- Learn,



innovate and build cutting edge innovations in the AI space collaborating with the best and brightest leading it.
- Ship current product features and improvements at a high velocity.
- Design, implement and deliver AI services to support product offerings for large-scale LLM serving.
- Collaborate closely with product management and partner teams to align technical direction with business goals.
- Coach and grow team members with design, code reviews, and mentorship serving as a role model leading product quality and velocity.
- Engage with customers to gather feedback and resolve complex issues.
- Innovate technical solutions, and patterns that will improve the availability, reliability, efficiency, observability, and performance of products.
- Embody our culture and values

Responsibilities- Build and scale the core serving systems and smart request routing and distribution for all LLMs (OpenAI, Anthropic, Mi

📌 Principal Group Engineering Manager (Hyderabad)
🏢 Microsoft
📍 Hyderabad