Description
???? ???? ?? ??? ??? ?? ??? ???????'? ???????
?????? ?????: ?? ??????? - ?? ??????? (?? ??? ??-?? ???)

Experience: 5+ yrs
Location: Bengaluru, Karnataka, India
Job Type: Full time

We are seeking a highly skilled Inference Engineer with strong expertise in Large Language Models (LLMs) and vLLM to build, optimize, and scale high-performance AI inference platforms. This role is ideal for professionals who are passionate about deploying production-grade generative AI systems, optimizing model serving, and improving inference efficiency for large-scale applications.
As an Inference Engineer, you will be responsible for designing, implementing, and maintaining scalable LLM inference infrastructure that delivers low-latency, high-throughput AI experiences. You will work closely with AI Researchers, Machine Learning Engineers, Platform Engineers, and Product teams to deploy state-of-the-art language models, optimize serving pipelines, and improve resource utilization across GPU-based environments.



This role offers the opportunity to work with cutting-edge AI technologies while solving complex performance, scalability, and infrastructure challenges.

Requirements
Key Responsibilities
- Design, deploy, and maintain scalable inference infrastructure for Large Language Models (LLMs) in production environments.
- Build and optimize high-performance model serving pipelines using vLLM and other modern inference frameworks.
- Improve inference latency, throughput, memory utilization, and overall system performance across GPU clusters.
- Deploy, monitor, and manage open-source and proprietary LLMs while ensuring reliability and scalability.
- Optimize GPU resource allocation, batching strategies, caching mechanisms, and model parallelism for efficient inference.
- Collaborate with Machine Learning Engineers to transition trained models into production-ready inference services.
- Develop APIs, microservices, and deployment workflows for AI-powered applications.
- Implem

📌 Inference Engineer (Bengaluru)
🏢 Weekday AI
📍 Bengaluru