This is one of the most technically novel roles in the team. You will build a pipeline that takes a Jira ticket or product specification and produces a first-draft Playwright test using Claude, and an autonomous agent that monitors the live test suite, detects flakiness, diagnoses root causes, and proposes or applies fixes. The quality of your agent layer will directly determine how fast the rest of the team can move.
WHAT YOU''LL OWN
Design and build the Claude-powered test generation pipeline: product spec or Jira ticket in, first-draft Playwright test out
Build the test maintenance agent: continuously monitors the suite, detects flaky or failing tests, diagnoses causes, and proposes or auto-applies fixes with human-in-the-loop review
Own the prompt engineering layer: maintain the prompt library, build an evaluation harness, and score generated test quality systematically
Define the acceptance workflow: which AI-generated tests require engineer review before merge, and which meet the bar for automatic acceptance
Instrument the agent layer: track cost per test generated, acceptance rate, time-to-fix for maintenance interventions, and overall flakiness reduction
Partner closely with automation engineers to understand product domain nuance and improve generation quality for complex business logic
Stay current with quick-moving LLM tooling and evaluate recent approaches that could improve speed or quality
MUST HAVE
Production experience with the Anthropic Claude API or OpenAI API specifically tool use, multi-step agent patterns, and structured output
Ability to evaluate LLM output quality rigorously: you build evals, not just vibes checks
Experience designing agentic systems that handle failure gracefully: retries, fallbacks, and human escalation paths
Comfort with ambiguity and incomplete requirements you are building something that does not exist yet in most organizations
NICE TO HAVE
Playwright or comparable E2E test framework experience you don''t need to be an expert but you must be able to read and reason about test code fluently
Experience with test infrastructure: fixture factories, seed scripts, workplace isolation
Prior work building agents with persistent memory, multi-tool orchestration, or complex retry logic
📌 Ai Automation Test Engineer Hyderabad
🏢 Keyworks Solutions
📍 Hyderabad
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.