Job Title: Automation Test Engineer with AI

Location: Chennai (Work from Home) Overlapping / Night shift

Experience: 3+

Responsibilities

- Agentic Benchmark Design: Generate, import, create, extend, and manage benchmarks for non-deterministic LLM agents, validating behaviour and responses.
- Functional & Regression Testing: Grow benchmarks and create new ones as new functionality or issues are uncovered.
- Benchmarking Automation Support: Contribute to streamlining the benchmarking process in alignment with product development work.

Collaboration:

Work closely with developers, product managers, and senior QA engineers to ensure test coverage aligns with product requirements.

- Defect Identification & Reporting: Log detailed defect reports and work with engineering teams to ensure timely resolution.
- Process & Traceability: Maintain requirement-to-test traceability and document detailed test plans and coverage to support audits/reviews for OEM programs.
- Release Reporting: Produce per-release quality reports as benchmarks evolve and grow over time.
- Performance & Load Testing Support: Assist in load and performance testing using tools such as JMeter.




- Agile Testing & Continuous Improvement: Participate in Agile development cycles, contributing to continuous process improvements.

Qualifications Basic Qualifications (Non-Negotiable)

- 3+ years of experience in software/product quality assurance.
- Confident usage of Bash and Python.
- Experience with manual testing and test case execution.
- Basic familiarity with automated testing frameworks such as Playwright or Selenium.
- Experience with testing web protocols (WebSocket message payload)
- Basic knowledge of load and performance testing (e.g., JMeter).
- Robust analytical and problem-solving abilities.
- Systems thinking.
- Familiarity with prompt evaluation, function-calling, and tool-use testing.

Preferred Skills

- Exposure to test automation scripting using Python.
- Experience with CI/CD integration.
- Familiarity with Conversational AI platforms and NLP-based applications.
- Experience with test management and result tracking.
- Experience testing conversational-agent systems and evaluating non deterministic outputs.

📌 AI Automation Engineer (Chennai)
🏢 Sensiple
📍 Chennai