About the role

At Networth Corp, we build the AI Gateway and LLMOps layer that lets enterprises deploy GenAI and agents at scale — one gateway, one policy, one bill, real observability. You'll work closely with our AI Red Team & Assurance leads so runtime enforcement matches the policies clients actually need.

The pain is real: every team ships their own LLM prototype, no shared gateway, no cost visibility, no guardrails, security teams saying no by default, and the "second brain" of the enterprise leaks context every third call. Your work makes GenAI production a routine event, not a launch.

What you'll do

- Stand up AI Gateway layers (Databricks AI Gateway, Portkey, LiteLLM, Kong AI Gateway, Azure API Management with AI policies) with unified auth, rate limiting, and cost attribution per team
- Deploy and operate LLM serving infrastructure — vLLM, TGI, Ray Serve, Databricks Model Serving, Azure ML endpoints, Bedrock — with canary rollouts, autoscaling, and traffic mirroring
- Configure guardrails at the gateway: PII redaction, prompt injection defense, jailbreak detection, output validation, policy enforcement (Guardrails.ai, NVIDIA NeMo Guardrails, Lakera, Protect AI, OWASP LLM Top 10)
- Enable agent platforms with governed tool-use, memory, sandboxed execution, identity propagation (LangGraph, CrewAI, Bedrock Agents, Azure AI Agent Service, Microsoft Copilot Studio, Databricks Mosaic Agent Framework)
- Wire evals into CI so model swaps and prompt changes are gated by regression harnesses (LangSmith, Braintrust, Ragas, DeepEval, Promptfoo, custom golden sets)
- Instrument LLM traffic: latency, token cost,



cache hits, quality drift, safety flags, jailbreak attempts (Langfuse, Arize Phoenix, WhyLabs, LangSmith, OpenTelemetry GenAI semantic conventions)
- Own RAG operations — chunking pipelines, embedding refresh, vector DB deployment, hybrid search tuning
- Build model gateway routing for cost / latency / quality tradeoffs — small model first, escalate to Claude Opus or GPT-5 only when needed
- Codify reusable LLMOps patterns Networth Corp deploys across clients

Stack

- Gateways & serving: Databricks AI Gateway · Portkey · LiteLLM · Kong AI Gateway · Azure APIM · vLLM · TGI · Ray Serve · Databricks Model Serving · Bedrock · Azure OpenAI
- Frameworks: LangChain · LlamaIndex · LangGraph · CrewAI · Semantic Kernel · Bedrock Agents · Mosaic Agent Framework
- Evals: LangSmith · Braintrust · Ragas · DeepEval · Promptfoo · TruLens
- Guardrails: Guardrails.ai · NVIDIA NeMo Guardrails · Lakera · Protect AI · Rebuff · Vigil
- Observability: Langfuse · Arize Phoenix · WhyLabs · Helicone · OpenTelemetry GenAI · MLflow tracing
- Vector & retrieval: pgvector · Pinecone · Weaviate · Qdrant · Chroma · Databricks Vector Search · Azure AI Search
- Model registries & lifecycle:



MLflow · Databricks Unity Catalog · Weights & Biases · Comet
- Runtime: Python · FastAPI · Docker · Kubernetes · Ray · Modal · Baseten

What we're looking for

- 4+ years in ML engineering, MLOps, or backend systems, with recent LLM production work (not just POCs)
- Hands-on with at least one AI Gateway or LLM proxy layer in production
- Robust Python engineering — asyncio, streaming, backpressure — this is not a notebook role
- Familiarity with LLM observability: traces, quality metrics, cost attribution, drift detection
- Comfort with guardrails frameworks and prompt-injection defenses
- Experience with agent frameworks in production — governed tool-use, not toy demos
- Understanding of the cost / latency / quality trade space across model families (Anthropic, OpenAI, Google, Meta, Databricks, Mistral)

Bonus

- OSS contributions to LLMOps, evals, or agent tooling
- Databricks AI Gateway or Bedrock production experience at enterprise scale
- Speaker at Data+AI Summit, MLOps World, LlamaCon, AI Engineer Summit, or similar
- OWASP LLM Top 10, NIST AI RMF, or MITRE ATLAS working knowledge
- Regulated-industry LLM deployment
- Anthropic MCP (Model Context Protocol) or A2A protocol experience

Skills tags

LLMOps · MLOps · Generative AI · LLM · AI Gateway · Databricks AI Gateway · LiteLLM · Portkey · Model Serving · vLLM · Agent Frameworks · LangGraph · Guardrails · Prompt Injection Defense · RAG · Vector Search · LangSmith · Langfuse · MLflow · OpenTelemetry · Python · AI Governance

📌 LLMOps Engineer / AI Gateway Specialist (India)
🏢 Networth
📍 India