SDET Lead – AI Center of Excellence (Shared Resource)

Role Overview

We are looking for an experienced SDET Lead (Software Development Engineer in Test) to support the AI Center of Excellence (AI CoE) across multiple product teams, AI accelerators, proof of concepts (POCs), and client engagements.

The role will drive quality engineering practices, establish reusable test automation frameworks, and ensure the reliability, performance, security, and functional correctness of AI-enabled applications. As a shared resource, the SDET Lead will collaborate with product managers, AI engineers, software developers, architects, and delivery teams to embed quality throughout the software development lifecycle.

The ideal candidate combines strong test automation expertise with an understanding of AI/LLM application testing and the ability to establish scalable quality engineering practices across teams.

Key Roles and Responsibilities

- Quality Engineering Strategy and Governance

- Define and implement a quality engineering strategy, testing standards, and best practices across AI CoE initiatives.
- Establish standardized test plans, test case design, defect management, and release readiness criteria.
- Integrate testing activities into Agile ceremonies, CI/CD pipelines, and release governance processes.
- Define quality metrics, including test coverage, automation coverage, defect leakage, execution trends, and release quality.
- Drive root cause analysis and continuous improvement across teams.

2. Test Automation Framework Development
- Design, develop, and maintain scalable, reusable test automation frameworks for web applications, APIs, and backend services.
- Establish automation standards, reusable libraries, test data management, and reporting mechanisms.
- Implement UI automation using tools such as Playwright or Selenium and API automation using appropriate frameworks.
- Integrate automated tests into CI/CD pipelines to enable continuous testing and faster feedback.
- Identify opportunities to reduce manual testing effort and improve regression testing efficiency.

3. AI and LLM Application Testing
- Develop testing strategies for AI-enabled applications, including RAG systems, AI agents, knowledge hubs, and LLM-powered accelerators.
- Validate AI outputs for correctness, relevance, groundedness, consistency, and adherence to expected business rules.
- Establish evaluation approaches for hallucination rate, precision, recall, F1 score,



and task success rate, as applicable to the use case.
- Test retrieval quality, document ingestion, context handling, prompt behavior, tool invocation, and agent workflows.
- Implement regression testing for prompts, models, retrieval configurations, and changes to AI application workflows.
- Collaborate with AI engineers to build evaluation datasets, test scenarios, and repeatable validation pipelines.

4. Performance, Reliability, and Security Testing
- Plan and execute performance, load, stress, and scalability testing for AI applications and supporting services.
- Validate API response times, concurrency, system stability, and resource utilization.
- Test failure scenarios, retries, timeouts, rate limits, and recovery mechanisms in AI workflows.
- Collaborate with development and security teams to identify application vulnerabilities and data-handling risks.
- Support validation of authentication, authorization, data privacy, and access controls.

5. Shared Resource Support Across AI CoE Teams
- Provide SDET leadership and testing support across multiple AI CoE products, accelerators, POCs, and client projects.
- Assess testing needs, prioritize assignments, and allocate effort based on delivery timelines, business risk, and project criticality.
- Engage early in solution design and requirements discussions to identify testability gaps and quality risks.
- Support teams during development, integration, client demonstrations, pilot releases, and production readiness.
- Create reusable testing assets and share knowledge across teams to avoid duplication of effort.
- Balance immediate project delivery needs with the long-term development of common testing capabilities.

6. Team Collaboration and Technical Leadership
- Mentor developers and QA engineers on automation, test design, and quality engineering practices.
- Conduct reviews of test strategies, automation code, and test coverage.
- Work closely with Product Managers, AI Engineers, Architects, DevOps, and delivery leads.
- Communicate quality risks, testing progress,



and release recommendations to stakeholders.
- Promote a shift-left testing culture and continuous quality ownership across engineering teams.

Required Skills and Experience
- 10+ years of experience in software testing, test automation, and quality engineering, with relevant experience leading testing initiatives.
- Strong programming skills in Python, TypeScript/JavaScript, Java, or C#.
- Hands-on experience with UI automation frameworks such as Playwright or Selenium.
- Solid experience in REST API testing, integration testing, and backend validation.
- Experience building reusable automation frameworks and integrating tests into CI/CD pipelines.
- Proficiency in Git, test management, defect tracking, and CI/CD tools.
- Experience with performance testing tools such as JMeter, k6, or equivalent.
- Strong understanding of Agile/Scrum, SDLC, test planning, and release management.
- Experience mentoring engineers and coordinating testing across multiple teams.

Preferred Skills
- Experience testing AI/ML applications, LLM-based solutions, RAG pipelines, or agentic AI workflows.
- Familiarity with LLM evaluation frameworks and tools such as DeepEval, Ragas, or equivalent.
- Understanding of vector databases, embeddings, prompt engineering, retrieval pipelines, and model evaluation.
- Experience with cloud platforms such as AWS, Azure, or GCP.
- Familiarity with observability, distributed systems, and AI application monitoring.
- Experience supporting multiple products or client engagements in a shared-services or Center of Excellence model.

Success Measures
- Increased automated test coverage and reduced manual regression effort.
- Improved defect detection before release and reduced production defect leakage.
- Reusable automation frameworks and test assets adopted across AI CoE teams.
- Reliable and repeatable evaluation of AI application quality.
- Faster feedback cycles through CI/CD-integrated testing.
- Improved release confidence, performance, and production readiness.
- Effective allocation of SDET capacity across multiple initiatives.

Role Expectations The SDET Lead is expected to be a hands-on technical leader, not solely a test coordinator. The role will establish reusable quality engineering capabilities, provide focused testing support to multiple AI CoE teams, and help ensure that AI solutions are demonstrably reliable, measurable, and ready for client adoption.

📌 SDET Lead (Hyderabad)
🏢 Sails Software
📍 Hyderabad