Title QA Engineer
Location India (Remote)
Duration Contract 6 to 12 months
Job Description
Overview
The QA Engineer II plays a crucial role in ensuring the quality and reliability of our Conversational AI solutions. Unlike deterministic software, the AI agents under test are non-deterministic, so benchmarking emphasizes semantic/behavioral evaluation in addition to exact-match assertions.

The ideal candidate has a solid foundation in software testing methodologies, experience with test case design and execution, and familiarity with automation frameworks. This role is ideal for detail-oriented professionals who are eager to learn, contribute to test automation, and help drive continuous improvements in software quality.

Responsibilities
- Agentic Benchmark Design: Generate, import, create, extend, and manage benchmarks for non-deterministic LLM agents, validating behavior and responses.
- Functional & Regression Testing: Grow benchmarks and create new ones as new functionality or issues are uncovered.
- Benchmarking Automation Support:



Contribute to streamlining the benchmarking process in alignment with product development work.
- Collaboration: Work closely with developers, product managers, and senior QA engineers to ensure test coverage aligns with product requirements.
- Defect Identification & Reporting: Log detailed defect reports and work with engineering teams to ensure timely resolution.
- Process & Traceability: Maintain requirement-to-test traceability and document detailed test plans and coverage to support audits/reviews for OEM programs.
- Release Reporting: Produce per-release quality reports as benchmarks evolve and grow over time.
- Performance & Load Testing Support: Assist in load and performance testing using tools such as JMeter.
- Agile Testing & Continuous Improvement: Participate in Agile development cycles, contributing to continuous process improvements.

Qualifications
Basic Qualifications (Non-Negotiable)
- 3+ years of experience in s

📌 QA Engineer (India)
🏢 ArrowCore Group
📍 India