Description

You should have

Real experience evaluating LLM and AI systems in production.

Fluency with agent architectures, prompting, and tool/function calling.

Strong Python and comfort with tracing/observability tooling (OpenTelemetry, Logfire or similar).

Judgment on model cost versus quality at scale.

Nice to have: LiteLLM/pydantic-ai, Temporal or similar workflow engines, MCP.

Requirements

We ship LLM-powered agents and need someone who owns whether they actually work well. You’ll define how we measure and observe model and agent behavior across the platform.

What you’ll do

Build our evaluation framework: offline evals, regression suites, and online quality signals for agents and the LLM gateway.

Own observability of model behavior: what the agent did, why, where it drifted, and what it cost.

Drive prompt, skill, and agent-role design and the model selection and cost/quality tradeoffs behind them.

Turn production failures into eval cases so quality compounds.

Job responsibilities

You should have





Real experience evaluating LLM and AI systems in production.

Fluency with agent architectures, prompting, and tool/function calling.

Strong Python and comfort with tracing/observability tooling (OpenTelemetry, Logfire or similar).

Judgment on model cost versus quality at scale.

Nice to have: LiteLLM/pydantic-ai, Temporal or similar workflow engines, MCP.

What we offer

Exciting Projects: We focus on industries like High-Tech, communication, media, healthcare, retail and telecom. Our customer list is full of fantastic global brands and leaders who love what we build for them.

Team-oriented Environment: You Can expand your skills by collaborating with a diverse team of highly talented people in an open, laidback environment — or even abroad in one of our global centers or client facilities

Work-Life Balance: GlobalLogic prioritizes work-life balance, which is why we offer flexible work schedules, opportunities to work from home, and paid time off an

📌 Senior AI/ML Engineer, Model Behavior and Evaluation6 (Bengaluru)
🏢 GlobalLogic
📍 Bengaluru