About the Role:

We are looking for a Senior Automation Test Engineer to own the end-to-end testing strategy across UI, APIs, and CI/CD pipelines. The role involves building reliable automation frameworks, evaluating AI-generated responses, and establishing trusted pass/fail signals to ensure release quality.

Key Responsibilities:

- Define and implement end-to-end testing strategies covering unit, API, UI, contract, smoke, and release-gate testing.
- Develop and maintain robust automation frameworks using Playwright or equivalent tools, along with API testing frameworks.
- Design AI response testing using golden datasets, prompt/playbook regression tests, structured-output validation, and grounding/citation assertions.
- Build evaluation harnesses for LLM and AI workflows, including scoring, thresholds, failure triage, and regression gates.
- Implement retrieval-quality checks wherever required by the product, without owning the data platform.
- Integrate automated test suites into CI/CD pipelines with a focus on reducing flaky tests and providing clear quality dashboards.




- Mentor team members on risk-based test coverage and maintainable test fixtures for tenant-based or multi-setting data.

Required Skills & Experience:

- B.Tech degree.
- 10 years of experience in QA / SDET, including ownership of end-to-end product testing strategy at a senior level.
- Strong programming skills in TypeScript, C#, or Python for test automation and evaluation frameworks.
- Hands-on experience evaluating LLM / AI outputs using golden datasets, evaluation rubrics, and regression gates.
- Experience with CI/CD tools such as Azure DevOps or GitHub Actions.
- Strong experience in API and UI automation at scale.
- Proven ability to improve flaky test suites and establish reliable release-quality signals.

Nice to Have:

- Experience testing RAG or hybrid search quality.
- Exposure to OWASP security smoke testing and performance smoke testing within CI/CD pipelines.
- Experience working with ephemeral and seeded test environments.

📌 Senior Automation Test Engineer - Playwright Testing (India)
🏢 BIG Ideas Social Media Recruitment
📍 India