Job Title: MLOps / Serving Engineer
Experience : 5+ years
Location : Hyderabad OR Pune
Notice Period: 0-30 days
Work mode - Hybrid

We are seeking an experienced MLOps / Serving Engineer who can design and operate the production serving infrastructure for fine-tuned LLMs on AWS — optimised inference engines, shadow-mode and staged rollout pipelines, monitoring dashboards, and the path from experimental model to full production traffic.
Key Responsibilities:
Deploy fine-tuned LLMs using vLLM, TensorRT-LLM, or Triton with continuous batching on AWS GPU instances
Build shadow-mode deployment: run fine-tuned model alongside production, log comparison data without impacting live traffic
Execute staged rollout: canary (5%) → gradual ramp (25% → 50% → 100%) with automated rollback on quality degradation
Optimize inference for input-heavy workloads (~17K token inputs, ~130 token outputs): prefill throughput, KV-cache, INT8 quantization
Build monitoring dashboards: latency, throughput, accuracy metrics, cost per request




Design auto-scaling; implement high-availability (2× instances); automated rollback triggers on end-to-end quality metrics

Requirements
5+ years MLOps or ML infrastructure engineering
Hands-on with vLLM, TensorRT-LLM, or Triton Inference Server
Deep familiarity with g5, p4de, p5 instance families, EC2 auto-scaling
Have worked on Deployment patterns like Shadow-mode, canary, A/B traffic routing, automated rollback
Experience onto Continuous batching, INT8 quantization, KV-cache management
Expertise on Docker, Kubernetes (EKS) for ML workloads
Worked on CloudWatch, Prometheus, Grafana

Benefits
Comprehensive Medical Coverage:
Health insurance of INR 5.0 Lakhs for you and your family (up to 6 members), ensuring complete peace of mind.
Robust Protection Plans:
Group Personal Accident Insurance and Group Term Life Insurance to safeguard you and your loved ones.
Retirement Benefits:
PF and Gratuity provided as per standard government regulations.
Versatile Work Opt

📌 MLOps / Serving Engineer (Hyderabad)
🏢 DataEconomy
📍 Hyderabad