Role: AI / Senior LLM Platform Engineer

Job Summary

We are looking for a Senior AI / LLM Platform Engineer to design and build scalable enterprise solutions, multi-model orchestration, intelligent model routing, and cost-effective AI capabilities. The engineer will focus on optimizing LLM selection, performance, token consumption, security, and reliability across enterprise agentic applications.

Key Responsibilities

- Design and implement context-aware Multi-LLM Orchestration and intelligent model routing.
- Integrate multiple enterprise-approved LLMs through centralized AI gateways.
- Optimize token consumption, context windows, prompt caching, and LLM execution costs.
- Develop model-selection strategies based on task complexity, quality, latency, availability, and cost.
- Implement LLM observability, evaluation, performance monitoring, and usage analytics.
- Build reliable LLM integrations with fallback, retry, rate limiting, and structured-output validation.
- Establish enterprise AI governance, security guardrails, and reusable LLM integration standards.
- Collaborate with AI engineering teams to integrate the LLM platform with multi-agent systems.





Required Technical Skills & Tools:

- Programming: Python, FastAPI, REST APIs, Async Programming
- LLM Platforms: Azure OpenAI, Claude, Gemini, OpenAI or equivalent enterprise models
- LLM Gateways: Portkey AI Gateway, LiteLLM or equivalent
- AI Frameworks: LangChain, LangGraph, LLM APIs
- Model Orchestration: Multi-LLM Routing, Model Selection, Fallback Strategies
- Token & Cost Optimization: Tokenization, Context Engineering, Prompt Caching, LLM Cost Management
- RAG: Embeddings, Vector Search, PostgreSQL/pgvector
- Observability: LangSmith, OpenTelemetry, Prometheus, LLM Evaluation Frameworks
- Cloud & Infrastructure: Microsoft Azure, AKS, Docker, Kubernetes
- Distributed Systems: Apache Kafka (preferred), Temporal (highly preferred)

Preferred Qualifications:

- Hands-on experience with multi-model routing and cost optimization.
- Knowledge of LLM security, prompt-injection prevention, and data governance.
- Experience establishing shared AI platform capabilities across multiple applications.

📌 Senior Engineer ML (Bengaluru)
🏢 Albertsons Companies India
📍 Bengaluru