Job Description
The AI Operations Lead owns how AI runs at QAD after go-live. You will run AI FinOps (where the money goes, whether it is worth it, and how to keep spend inside budget), build the observability that tells us when an agent is drifting, failing or getting expensive, and stand up the operational support model that colleague across the business rely on when something breaks or they need help.
This is an operations and service ownership role, not a reporting role. Dashboards are an input, not the output. You will be measured on AI that stays reliable, affordable and trusted in production. If spend runs ahead of value, an agent quietly degrades after a model change, or a function cannot get support, that is your problem to solve, not to report.
Where this role sits
- Reports to: Head of the AI Centre of Excellence
- Works with: AI Engagement Specialists, Lead AI Architect and Senior AI Engineer daily; QAD Engineering teams, External implementation partners and AI vendors; Finance, Procurement, IT, Security, GRC and Legal as required
- Owns: AI consumption and cost governance, production observability, AI service and incident management orchestration, agent registry and publish gate operations, model refresh operations, and partner and vendor service levels
- Accountable for: Reliable, affordable, supported AI in production, measured on spend against plan, cost per transaction, service levels and user satisfaction, not ticket counts
What you will own
AI FinOps and consumption governance
- Own AI spend end to end across licences, usage-based model consumption, cloud inference and partner run costs: forecast, budget, track and explain the variance every month with Finance.
- Run the entitlement and consumption model: persona caps,
top-ups and tier upgrades, kept in line with real usage. Reclaim idle licences and stop paying for tools nobody uses.
- Make unit economics visible for every live workflow (cost per transaction, per user and per outcome) and work with the architect on routing, caching and model-tier choices. Most overspend is a frontier model doing routine work.
- Bring usage evidence to commercial decisions: showback to functions, and renewal, commitment and tier reviews with Procurement and the vendors.
Observability and production performance
- Build and run observability for the platform and every live agent: traces, latency, error and failure rates, token usage and cost, tool-call success, guardrail triggers, escalation and human-override rates.
- Run evaluations in production with engineering: scheduled evaluation runs against curated datasets, drift and regression detection after model or prompt changes, and alert thresholds that someone acts on.
- Own model and vendor change: plan model refreshes and deprecations, regression-test before switching, and keep a rollback path for every change.
- Give leadership one operational view: service health, spend and quality for the Head of AI and function leaders, with a weekly operations review.
Operational support to the business
- Stand up AI service management with IT: intake, a tiered support model, triage,
service levels and escalation to engineering and partners.
- Own AI incident and problem management, including AI-specific incidents: wrong or harmful output, data exposure, runaway cost and prompt injection. Run post-incident reviews and see corrective actions through.
- Support colleagues across functions with access, entitlements, connectors and usage questions, and keep the knowledge base and tool guidance current so most questions never need a ticket.
- Feed operational insight to the Engagement Specialists: which workflows are struggling on quality, adoption or cost, with the evidence.
CoE operating model and controls
- Run the agent registry and publish gate day to day: intake, checks, approval evidence, named owners, review dates and retirement.
- Build operational requirements into every PRD: logging, cost tagging, evaluation hooks, service levels and runbooks are acceptance criteria, agreed before build, not added after go-live.
- Maintain operational controls with Security, GRC and Legal: access reviews, audit trails, data-handling checks and evidence for audits.
- Hold partners and vendors to their service levels: run cost, availability, quality and reporting, with regular service reviews.
- Run the CoE operating cadence and reporting to the Head of AI: a monthly AI operations and spend report and a quarterly business review.
Continuous improvement
- Automate the operation: use scripting and AI to automate monitoring, reporting and triage so the team scales with the number of live agents, not with headcount.
- Turn incidents into standards: reusable runbooks, alert patterns and cost controls that every current workflow inherits.
📌 AI Operations Lead (Pune)
🏢 QAD
📍 Pune