Trintech is an award-winning AI-driven Fintech SaaS organisation transforming
the way finance teams operate.
We are looking for a Senior Automation QA Engineer to define and execute test
strategies for AI agent features in the Agent Development Squad. This role goes
beyond standard API and UI automation. You must understand the unique challenges
of testing non-deterministic LLM outputs, multi-step agent graphs, and
audit-critical financial workflows. Pytest and Playwright are the primary
tooling standards. LLM and agent testing awareness is required. Hands-on
production experience with Langfuse or LLM eval harnesses can be developed on
the job.
What You'll Do
* Design and implement test automation strategies for backend APIs, agent
orchestration logic, and Angular-based agent UIs.
* Build and maintain Pytest-based test suites for Python backend services and
agent workflow validation.
* Implement Playwright automation for Angular-based UI, covering interaction
flows, approval queues, and streaming response handling.
* Validate multi-step agent workflows by asserting on intermediate states, tool
call outputs, and handoff results, not just final API responses.
* Use Langfuse traces to validate agent reasoning chains, confidence scores,
and execution paths.
* Build and maintain LLM evaluation harnesses covering prompt regression
testing, RAG output validation, and model version change detection.
* Integrate test suites into Azure DevOps CI/CD pipelines as automated quality
gates.
* Verify audit log integrity, including immutable log structure, correlation ID
chains, and SOX-compliant event emission.
Who You Are
Test Automation - Primary Gate
* 5+ years of QA automation experience for API and web-based applications.
* Pytest as the primary test framework for Python backend and agent logic
validation.
* Playwright for UI automation targeting Angular-based interfaces.
* REST API testing using Postman, HTTPX, or equivalent tools for contract and