Role Overview

You will be the Quality Intelligence Architect of Exterview.

As a QA Automation Engineer (10+ Years), you’ll design, implement, and own the automation and quality orchestration system that validates AI-driven candidate screening, live interviews, scoring agents, and feedback pipelines.

This is quality engineering at scale: validating probabilistic AI outputs, real-time interview flows, event-driven systems, and media-heavy workflows across thousands of concurrent interviews.

You’ll work closely with Backend, AI Engineering, Prompt Engineering, and Frontend teams to ensure quality is built into the system, not tested after the fact.

Execution is tracked via Linear (tasks), Notion (PRDs & test strategy), and GitHub (automation reviews) ensuring quality decisions are transparent and measurable.

Key Responsibilities

Automation Architecture

- Design and own a scalable, maintainable automation framework for:

- Web (Next.js)

- APIs (GraphQL via AppSync, REST)

- Event-driven and async workflows

- Move QA from test execution to quality architecture ownership.

AI & Probabilistic System Validation
- Validate:

- Resume parsing accuracy

- AI scoring consistency

- Interview logic stability

- Define assertion strategies for non-deterministic AI outputs.

Real-Time & Media Workflow Testing
- Test live interview systems:

- VideoSDK + Tavus video flows

- Real-time state updates

- Validate video upload, playback, retries, and failure handling.

API, Event & Integration Testing
- Build API-first automation for:

- GraphQL queries & mutations

- Lambda & microservice APIs

- Validate event-driven flows (notifications, interview state transitions).

Performance, Load & Reliability Testing




- Design load and stress tests for:

- Interview orchestration APIs

- Video playback & report generation

- Ensure system reliability under peak concurrency.

CI/CD & Quality Gates
- Integrate automation into GitHub Actions.

- Define release quality gates and production sign-off criteria.

- Track flaky tests, failure patterns, and quality metrics.

Cross-Team Quality Leadership
- Translate product requirements into test strategies.

- Partner with engineering to improve testability & observability.

- Mentor junior and mid-level QA engineers.

Success Metrics

First 90 Days
- Secure automation framework covering UI + API + AI workflows.

- Flaky test rate reduced to near zero.

- API-level load tests live for interview & video flows.

- Release quality gates enforced in CI/CD.

12 Months
- End-to-end automation coverage for all interview types.

- Predictable, fast release cycles with no QA bottlenecks.

- AI validation framework adopted as product standard.

- QA recognized as product quality owner, not gatekeeper.

Must-Haves

- 10+ years in QA Automation / SDET roles

- Strong experience designing automation frameworks, not just writing scripts

- Expertise in API testing (GraphQL & REST)

- Experience testing real-time and async systems

- Ability to validate AI-driven, non-deterministic outputs

- Strong understanding of CI/CD,



release quality, and production readiness

Nice-to-Haves

- Experience testing AI/ML or agent-based systems

- Media or video workflow testing experience

- Startup or high-scale SaaS background

- Prior ownership of QA strategy for a product

Tech Stack Visibility

- Frontend: Next.js, TypeScript

- Automation: Cypress, Playwright, Jest

- Backend: Node.js, AWS Lambda, Serverless Framework

- APIs: GraphQL (AWS AppSync), REST

- Auth: AWS Cognito (OAuth, OTP)

- Data: MongoDB Atlas

- Media: VideoSDK, Tavus, Amazon S3

- Communication: Twilio (SMS, WhatsApp, IVR)

- Observability: Langfuse, internal monitoring

Assessment (PoC)

Objective

Validate your ability to design quality systems, not just write test cases.

Challenge Overview

Design and implement an automation strategy for an AI-driven interview flow.

The system should validate:

- Resume upload & parsing

- AI resume–job match scoring

- Interview scheduling

- Live interview execution

- Interview report generation

Functional Expectations1. Automation Design
- Cover:

- One UI flow

- One GraphQL API flow

- One real-time or async workflow

- Show clear separation between test layers.

2. AI Validation Strategy
- Define how you:

- Assert AI scores

- Detect drift or instability

- Handle acceptable variance

3. Performance & Reliability
- Include:

- One API load test

- Failure / retry validation

- Avoid direct S3 URL testing; use APIs only.

Deliverables
- Automation repo (clean structure)

- Test strategy README

- Example AI assertion logic

- CI pipeline config (GitHub Actions)

- ≤ 5 min Loom walkthrough explaining decisions

📌 QA automation engineer (Hyderabad)
🏢 Exterview
📍 Hyderabad