Our Purpose

Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we’re helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, easy, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential.

Title and Summary

Senior AI Product Ops

AI Ops
Overview:
Mastercard's Operational Intelligence team is building the next generation of data products and AI solutions that help customers access and use operational insights in current ways. As AI agents move from POC to production, their behavior keeps changing in real time: the context they retrieve, the skills they select, how they resolve exceptions. These agents are never really “finished.” They keep learning from production outcomes, so their reliability, safety, and trustworthiness need to be actively operated, not just monitored.



That is the job of AI Ops: making sure each agent’s reasoning stays correct, its context stays current, and its behavior stays governed as it learns.
Role:
• Own end to end production monitoring for one or more AI agents, tracking key metrics, KPIs, accuracy, latency, SLA breaches, guardrail triggers, and out of scope rates through dedicated observability dashboards
• Serve as L1/L2 production support, providing first response monitoring, triage, and escalation to tech teams as current agentic services go live
• Identify and diagnose issues by monitoring eval scores and drift alerts, inspecting failing traces and patterns, and classifying root causes such as intent, tool, parameter, or hallucination errors
• Evaluate whether the agent’s reasoning remains correct in production, checking whether context retrieval stays current, whether resolutions match ground truth, and whether the learning l

📌 Senior Ai Product Ops Pune
🏢 Mastercard
📍 Pune