Job Summary:

Summary:

We are looking for a Software Test Automation Lead to own test automation standards and QA practice across the AI Platform. You sit in the Platform Team and have two equally important responsibilities. First, you own the quality assurance of the Platform Teams own services multi-tenant architecture, IAM infrastructure, event-driven backbone, API gateway, billing and metering engine, and the agent marketplace working hands-on alongside the Senior QA Engineer embedded in the Platform Team. Second, you own the QA practice across all Agent Stream pods standardising test frameworks, tooling, quality gates, and evaluation approaches across 4-5 agent development squads, and growing QA capability across the stream. This is a hands-on IC role with stream-wide practice influence. You lead through standards and example, not through line management. The right person has deep SaaS platform testing experience, owns test framework design independently, and is genuinely curious about building LLM and agent testing capability in a space where industry playbooks are still being written.

What Youll Do

Platform Team Quality Assurance

- Own test automation for Platform Team services — multi-tenant data isolation, IAM and OAuth 2.0 flows, event-driven backbone integrity, API gateway contract testing, billing and metering accuracy, and agent marketplace functionality.
- Design and maintain test strategies for complex SaaS platform concerns — tenant boundary validation, cross-tenant security isolation, event ordering and delivery guarantees, and immutable audit log integrity.
- Build and maintain Pytest-based test suites for Platform Team Python services, validating API contracts, event schemas, and service-level behaviour under multi-tenant conditions.
- Collaborate with the Platform Software Architect and Senior QA Engineer on test coverage strategy — ensuring platform capabilities are testable by design and quality gates are integrated into the Platform Team’s CI/CD pipelines.
- Own audit trail validation for platform services — immutable log structure, correlation ID chains, and event emission correctness that underpin compliance requirements.

QA Practice Leadership Across Agent Stream

- Define and own QA standards across all Agent Stream pods — test framework patterns, Pytest and Playwright conventions, CI/CD quality gate requirements, and test code quality bar that all pod QA engineers follow.
- Standardise LLM and agent testing approaches across pods — define how non-deterministic outputs are handled, what constitutes a regression, and how agent workflow validation is structured consistently across squads.
- Review and challenge test strategies proposed by pod QA engineers — raise the bar on coverage, framework design, and evaluation rigour across the stream.




- Drive LLM testing capability building across the stream — introduce and embed Langfuse trace validation, eval harness patterns, and prompt regression testing as shared practices, not squad-level experiments.
- Contribute to QA hiring — define the technical bar for QA engineers across the stream, participate in interviews, and ensure new hires are assessed consistently.

Quality Standards & Engineering Craft

- Set the test code quality bar for the AI Platform — clean, reusable, well-structured Pytest and Playwright implementations that others reference as the standard.
- Identify and address systemic quality gaps proactively — gaps in coverage, inconsistent practices across pods, or framework debt that is slowing squad delivery.
- Integrate test suites into Azure DevOps CI/CD pipelines as automated quality gates — ensuring consistent pipeline standards across Platform Team and Agent Stream.

Who You Are

Test Automation Engineering — Primary Gate

- 7+ years of software testing and automation experience, with primary depth in SaaS platform or backend services testing — not just feature-level API testing.
- Pytest at framework design depth — fixtures, conftest architecture, parametrization, test isolation, and plugin authoring. Owns framework decisions independently.
- Playwright (preferred) or Selenium at depth — Angular UI automation covering interaction flows, async state handling, approval queues, and streaming response validation. Tool depth matters more than the specific framework.
- Python for test framework development and custom assertion logic. REST API testing using Postman, HTTPX, or equivalent for contract and schema validation.
- Azure DevOps CI/CD quality gate integration — pipeline definition, test reporting, and quality gate enforcement across multiple squads.
- SQL and NoSQL for test data management, teardown, and database state validation at service level.

SaaS Platform Testing — Primary Gate

- Production experience testing multi-tenant SaaS systems — tenant isolation validation, cross-tenant security boundary testing, and scoped data access verification.
- Event-driven system testing — message ordering, delivery guarantees, idempotency, and event schema contract validation in asynchronous service architectures.
- API gateway and contract testing — OpenAPI schema validation, versioning, rate limiting behaviour, and inter-service contract testing.
- IAM and authentication flow testing — OAuth 2.0 flows, token lifecycle validation, and identity propagation across services.




- Audit and compliance trail validation — immutable log integrity, correlation ID chains, and event emission correctness for compliance-grade platform services.

QA Practice & Cross-Squad Leadership

- Experience defining and enforcing QA standards across multiple squads or teams — framework conventions, coverage requirements, and quality gate patterns adopted through influence, not mandate.
- Track record of raising QA capability in a team — through mentoring, framework contributions, standards documentation, or introducing new testing practices.

Awareness

- LLM and agent testing concepts — understands why standard assertion patterns must be adapted for non-deterministic outputs, multi-step agent graphs, and confidence-based validation. Hands-on production experience is not required; genuine curiosity and the ability to learn and lead in this space is.
- Langfuse or equivalent LLM observability tooling — what trace data contains and how it can be used for agent validation. Awareness is sufficient at hire; ownership is expected to develop in the role.

Nice to Have

- Hands-on Langfuse experience — trace validation, eval pipeline setup, and confidence score assertion.
- LLM evaluation frameworks — Ragas or equivalent for RAG output quality testing.
- Performance and load testing for API or LLM inference endpoints.
- Azure Monitor or Application Insights for observability-assisted debugging. Contributions to open-source test tooling or automation frameworks.
- Prior formal or informal QA leadership experience — tech lead, QA chapter lead, or equivalent.

What You’ll Learn & Gain

- Platform-level test ownership on a live enterprise AI product — defining what quality means for foundational SaaS infrastructure that multiple stream teams build on, where your test standards have direct consequences for the entire platform.
- The chance to build LLM and agent testing capability from the ground up across a multi-squad AI engineering organisation — a discipline that is still being defined at industry level, with the scope to shape how it is done
- Stream-wide practice leadership experience as a senior IC — influencing QA standards, tooling choices, and engineering quality across 4-5 agent pods without line management, which is a distinct and increasingly valued career track in engineering organisations.
- Close collaboration with the Platform Architect, Engineering Manager, Software Architect (Agent Stream), and pod Tech Leads in a fast-moving multi-stream product organisation where your quality standards are visible and impactful.

If you are a test automation engineer with deep SaaS platform testing experience, the technical confidence to define standards that others follow, and the curiosity to build LLM testing capability in a space that is still being defined — we encourage you to apply.

📌 Software Test Automation Lead AI Platform (Bengaluru)
🏢 Peoplefy Infosolutions
📍 Bengaluru