Overview
About the Role
AI decision systems are rapidly evolving. Modern platforms must combine deterministic logic, machine-learning models, advanced AI agents, and human expertise to make high-quality decisions at scale.
Evaluating these systems requires more than static benchmarks and traditional review queues. It requires platforms where automated evaluators, AI agents, production signals, and human experts work together continuously.
Microsoft Ads is looking for a hands-on Principal Software Engineer to build the next generation of agentic evaluation, audit, and human-review platforms.
You will create systems that help teams understand how decisions were made, identify where they fail, investigate supporting evidence, prioritize uncertain or high-impact cases, and turn review outcomes into measurable improvements.
The platform will connect:
production decisions → automated evaluation → agentic investigation → human judgment → system improvement

Responsibilities
Architect and build scalable platforms for agentic evaluation, production audits, and human-in-the-loop review.




Develop workflows in which AI agents gather evidence, analyze cases, compare decisions, and assist reviewers while preserving human oversight and accountability.
Build reviewer experiences that present the right evidence, explanations, uncertainty, and recommended actions at the right time.
Design distributed orchestration systems for long-running workflows involving models, agents, tools, automated checks, and human decisions.
Provide technical leadership across engineering, applied science, product, design, operations, and policy teams.

Qualifications
Required Qualifications
Bachelor’s or Master’s degree in Computer Science, Engineering, or a related technical field, with 8+ years of relevant industry experience.
Robust production-coding skills in at least one of C++, C#, Java, Python, or TypeScript.
Demonstrated experience designing, building, and operating large-scale distributed services

📌 Principal Software Engineer (Hyderabad)
🏢 Microsoft
📍 Hyderabad