Job Title: QA Automation Engineer

Location: HYD Campus, Hyderabad

Years of Experience: 5-7 Years

Job Summary: We are seeking a skilled QA Automation Engineer with a strong background in AI Evaluation & Benchmarking. The ideal candidate will be responsible for developing and executing automated test scripts, ensuring the quality and performance of our AI-driven applications. You will work closely with cross-functional teams to identify testing requirements and deliver high-quality software solutions.

Responsibilities

- Design, develop, and maintain automated scripts for AI evaluations.
- Conduct AI evaluation and benchmarking to assess model performance and accuracy.
- Manually generate evaluation datasets or retrieve open source samples.
- As human rater score and evaluate the ouput generated by AI models.
- Provide competitive analysis of the product features vs similar 3rd party products.
- Participate in code reviews and provide feedback on testability and quality.

Mandatory Skills





- Strong knowledge of AI Evaluation & Benchmarking techniques.
- Familiarity with open source frameworks used for evaluating, testing and optimizing AI Agents and LLM, in isolated reproducible environment.
- Experience with programming languages such as Python, Java, or Go.
- Familiarity with CI/CD pipelines and version control systems (e.g., Git).

Preferred Skills

- Experience with Claude Code, Antigravity, Gemini Code Review Agent and Agentic Actions.
- Working experience with Gemini CLI extensions, MCPs and Memory.
- Experience in Agile/Scrum methodologies.
- Strong analytical and problem-solving skills.

Qualifications

- Bachelor's degree in Computer Science, Engineering, or a related field.
- 5-7 years of experience in QA Automation with a focus on AI applications.
- Proven track record of successful project delivery in a fast-paced workplace.

📌 Lead Software Engineer (Hyderabad)
🏢 Virtusa
📍 Hyderabad