Overview We are looking for an experienced QE III Senior Quality Engineer to lead quality validation for our agentic AI models product to assess compliance reasoning outcomes and evaluation status This role requires strong expertise in test automation using Selenium WebDriver CI CD enablement and solid experience validating AI-driven systems including model outputs confidence scores and decision traceability The QE III will act as a quality owner partnering closely with Product Engineering and Data Science teams Responsibilities AI Compliance Quality Ownership Own end to end quality strategy for the AI agent models Agentic AI platform covering UI backend services and AI model outputs Validate AI agent behavior including Decision correctness Evaluation status Confidence thresholds Explainability traceability of responses Define test approaches for non deterministic AI behavior golden datasets tolerance-based assertions confidence validation Partner with Data Science teams to validate model inference results regression drift and version-to-version consistency Automation Framework Development Design develop and maintain robust Selenium WebDriver automation frameworks Java preferred Implement automation for Web UI workflows Compliance dashboards Agent evaluation flows Admin configuration screens Ensure automation follows industry best practices Page Object Model POM reusable utilities proper wait strategies and stable selectors Page Object Model POM Data driven and keyword driven testing Reusable libraries and utilities Robust synchronization and wait strategies to reduce flakiness Integrate automation suites into CI CD pipelines using Maven Jenkins GitHub CI CD Quality Engineering Practices Own Jenkins pipelines for automated test execution smoke regression AI evaluation suites Enable automation execution as part of Pull request validation Nightly regression Release readiness checks Analyze test failures improve flakiness and ensure high signal to noise ratio in automated results Produce actionable test metrics coverage defect leakage automation ROI Functional Integration Regression Testing Drive functional integration and regression testing across multiple services and AI agents Validate API integrations REST Data flows across agent pipelines Evaluation status lifecycle queued processed evaluated completed Perform exploratory testing on new AI capabilities and edge cases Leadership Collaboration Act as QE SPOC for the pod product area Mentor junior engineers on automation testing strategies and AI validation approaches Participate in requirement reviews and proactively identify quality risks early in the SDLC Qualifications Core Experience 6-9 years of experience in Software Quality Engineering Robust hands on experience in test automation using Selenium WebDriver Proficiency in Java or similar OOP language for automation Solid experience working with Maven build dependency management Jenkins CI pipelines GitHub version control PR workflows AI Data Driven Systems Must Have Experience testing AI ML driven applications analytics platforms or rule decision engines Understanding of Model inference pipelines Confidence scores Evaluation metrics Handling non deterministic outputs Ability to design validation strategies for AI agents beyond simple pass fail testing Testing Tooling Strong experience with API testing REST TestNG JUnit Agile Scrum methodologies Familiarity with test management and defect tracking tools Azure DevOps TFS etc Experience validating complex workflows across UI API layers Nice to Have Experience in compliance audit or regulatory systems Exposure to performance testing e g JMeter for dashboards or AI pipelines Knowledge of logging monitoring and observability tools for troubleshooting AI behavior Experience validating data privacy and security aspects of AI systems

📌 Qa Engineer Iii (Telangana)
🏢 RealPage
📍 Telangana