Lead the agentic AI engineering function responsible for the design, architecture, and quality of all AI agents on the platform. Define engineering standards for prompt design, agent evaluation, and LLM integration across the team.

WHAT YOU'LL DO

- Architect the agent engineering framework: orchestration patterns, prompt governance, and tool-calling standards
- Lead design review of all agent implementations across the team
- Own the RAG pipeline architecture: embedding strategy, retrieval optimization, and accuracy benchmarks
- Define the agent evaluation framework: golden datasets, confidence calibration, regression testing
- Set prompt engineering standards and chair the prompt review board
- Mentor JL1 engineers on agent development and evaluation
- Monitor and optimize agent performance in production: latency, cost, and accuracy





REQUIRED SKILLS & EXPERIENCE 10+ years in AI/ML engineering; 5+ years with production LLM systems Deep expertise in prompt engineering, agent design, and LLM integration

Experience architecting RAG pipelines at scale Agent evaluation: hallucination testing, confidence scoring, regression frameworks

Python expert; TypeScript/Node.js familiarity a plus Experience leading AI/ML engineering teams

Understanding of LLM cost management, token budgeting, and latency optimization

NICE TO HAVE

- Experience with Model Context Protocol (MCP) or tool-calling standards
- Background in enterprise AI platform builds
- Knowledge of ITSM or IT operations domains

📌 AI ML Engineer (Pune)
🏢 Futran Tech Solutions
📍 Pune