Job highlights

- Minimum of 10 years experience working as Principal Engineer or Architect with recent 2-4 years of experience in building AI Tooling,Agentic testing frameworks,Eval Pipelines,Architecting real-time production data feedback loops Data Quality standards etc.

Job Summary

The Architect serves as a technical authority and strategic leader, driving the transformation into an AI-first Quality Engineering (QE) organization. This role is responsible for architecting and developing custom AI testing frameworks and proprietary AI tools designed specifically to test, evaluate, and validate AI-driven products, ensuring high reliability, performance, and scalability across enterprise systems.

What You'll Do

- AI Frameworks Tool Development: Design, architect, and build custom test automation frameworks and evaluation pipelines for golden data sets and real-time production data explicitly engineered to test non-deterministic AI models and AI-native applications.

- Custom AI Tooling Creation: Develop proprietary AI-driven tooling, including synthetic data generators, LLM evaluation harnesses, prompt regression suites, and self-healing test automation engines.

- Cross-Department Tooling Standards: Provide technical leadership on the design and adoption of AI test tools and automation strategies extending across multiple product departments.

- Strategic Architectural Direction: Lead critical, multi-dimensional initiatives that define the technical and business direction of quality engineering across the enterprise.

- Executive Technical Advice: Provide expert technical advice and counsel to executive leadership on broader company initiatives and AI safety/quality gates.





- Critical Problem Resolution: Solve complex design issues and critical technical bottlenecks that impact future products, AI capabilities, and business growth.

- AI Quality Practice Evolution: Continually evaluate current quality practices and invent new AI-first strategies to systematically test AI hallucinations, model drift, bias, and output accuracy.

- Cross-Discipline Expertise: Apply subject matter expertise across software architecture, machine learning pipelines, data engineering, and test automation.

- Groundbreaking Methods: Employ creative, groundbreaking testing methodologies and frameworks to evaluate next-generation AI features.

- Organizational Goal Setting: Help define long-term organizational goals, establishing ROI benchmarks for internal AI testing tools and framework adoption.

- Engineering Mentorship Influence: Provide specialized technical guidance, and mentorship to senior engineers across cross-functional teams and influence the adoption of AI tools across teams.

- Industry Representation: Build internal decision-maker networks and represent the company as a technical spokesperson at external AI and quality engineering industry forums.

What You Will Bring to Coupa





- Minimum of 10 years experience working as Principal Engineer or Architect with recent 2-4 years of experience in building AI Tooling, Agentic testing frameworks, Eval Pipelines, Architecting real-time production data feedback loops Data Quality standards etc.

- Architectural Experience: Proven experience in technical architectural leadership for complex enterprise platforms.

- Must-Have Technical Expertise: Advanced fluency in programming languages and deep knowledge of building the connective tissue between non-deterministic AI outputs and deterministic enterprise systems.

- AI Tool Framework Engineering: Hands-on experience shipping LLM-powered features and building custom test frameworks, model evaluation harnesses, or automated testing tools.

- Agentic Frameworks: Familiarity with or hands-on experience using open-standard agentic architectures (e.g., LangChain, AutoGen, MCP).

- The Builder Mindset: A strong drive to build side projects and the ability to move rapidly from a blank page to functional, pluggable AI components.

- Strategic Problem Solving: Strong problem-solving and analytical skills, including the ability to debug complex systems by relying on logs, traces, and evals rather than guessing.

- Technical Leadership: Experience leading cross-functional teams, managing stakeholders (exec-level, cross-region), and providing technical agility in an evolving environment.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

📌 Principal QE Engineer/Architect (Pune)
🏢 Coupa
📍 Pune