Key Skills: Automation Testing, Selenium

Roles and Responsibilities:

- Design, develop, and maintain automated test frameworks for AI-powered web applications, APIs, and backend services.
- Build automated test suites for LLM-based and agentic AI platforms, including functional, integration, regression, and end-to-end testing.
- Develop test strategies for non-deterministic AI systems using statistical validation, semantic similarity, and confidence-based assertions.
- Perform failure injection, resilience, and recovery testing for asynchronous and event-driven workflows.
- Validate multi-tenant security, data isolation, sandboxing, and access control through automated testing.
- Automate API testing, workflow validation, and release verification within CI/CD pipelines.
- Develop synthetic test data and reusable test harnesses to improve automation coverage and reliability.
- Execute performance, scalability, concurrency, and reliability testing for AI platforms.
- Collaborate with AI engineers, platform engineers, and product teams to define acceptance criteria and improve system quality.
- Mentor QA engineers and contribute to reusable automation frameworks and best practices.

Skills Required:

- 8+ years of experience in QA Automation, API Testing, AI/LLM testing, and automation framework development.
- Strong experience in Automation Testing and QA framework development.




- Hands-on expertise with Playwright, TypeScript, and Selenium.
- Experience with API Testing, backend validation, and integration testing.
- Strong understanding of AI/LLM testing, prompt validation, RAG systems, and non-deterministic application testing.
- Experience testing asynchronous and event-driven systems.
- Knowledge of CI/CD tools such as GitHub Actions, Jenkins, or Azure DevOps.
- Strong SQL skills and experience with data validation and synthetic test data generation.
- Experience with test automation frameworks, release gates, and quality metrics.
- Familiarity with security testing, adversarial testing, and OWASP LLM Top 10 concepts.
- Solid analytical, troubleshooting, and communication skills.

Good to Have:

- Experience with Temporal workflows or similar orchestration platforms.
- Knowledge of performance testing tools such as k6 or Locust.
- Familiarity with LangSmith, Langfuse, Promptfoo, DeepEval, or Arize Phoenix.
- Experience with RAG, OpenTelemetry, OCR, or document processing systems.
- Exposure to SOC 2, GDPR, accessibility testing, or qTest.
- Experience testing multi-tenant SaaS platforms and AI agent frameworks.

Education: Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.

📌 AI QA Automation Engineer (Bengaluru)
🏢 Careernet
📍 Bengaluru