Work Location: Coimbatore

Work Mode: Hybrid/Remote

Notice Period: 0 to 30 days

Experience:

(4+) years experience with QA Automation (1+ year with Gen AI Testing)

Qualifications:

- Bachelor s degree in Computer Science, Engineering, Mathematics, or related field.
- Proficiency in programming languages such as Python (preferred) or Java.
- Strong understanding of testing principles and methodologies.

We are seeking a skilled QA Gen AI Automation Engineer to join our team. The ideal candidate will have strong hands-on experience in Python, Generative AI, and automated testing.

In this role, you will test and validate AI-driven solutions, automate regression tests, and ensure the performance and functionality of applications in an Agile environment.

Key Responsibilities:

Automation Testing:

- Develop and maintain automated regression tests using tools like Selenium WebDriver/Playwright and pytest.
- Conduct API testing using tools such as Postman.

Generative AI Testing:

- Design and execute test cases for Generative AI models and applications.
- Evaluate the performance and quality of Generative AI outputs, ensuring they meet functional requirements and quality standards.
- Validate AI systems such as Chatbots, RAG applications, and LLM-based workflows.
- Perform LLM evaluation using frameworks such as DeepEval and RAGAS.

Programming Scripting:

- Write and maintain automated test scripts in Python.
- Work with XML, JSON, and other scripting languages to create efficient testing environments for AI systems.

Testing Tools Frameworks:

- Utilize JMeter for performance testing and load testing.
- Implement and manage Continuous Integration (CI)



pipelines using tools such as Jenkins and Bamboo.

Database SQL:

- Hands on experience with Panda Data frames
- Execute SQL queries for database testing (inserts, updates, joins, etc.).
- Test non-GUI applications (e.g., SQL, flat files, XML).
- Work with Vector Databases such as OpenSearch or similar vector DB technologies.

Agile DevOps:

- Collaborate within an Agile (Scrum) framework for efficient testing and development.
- Utilize Atlassian JIRA and Confluence for project tracking and documentation.

Product Evaluation Troubleshooting:

- Identify and troubleshoot product issues, ensuring quality and stability.
- Evaluate product performance and provide feedback to development teams.

Web Application Testing:

- Conduct testing for web applications and web services (REST, SOAP).

Required Skills Experience:

Programming:

- Proficient in Python for writing test scripts and automating processes.

Automation Testing:

- Experience with Selenium WebDriver/Playwright and pytest for automation testing.
- Hands-on experience in API testing using tools like Postman.

Generative AI AI Testing:

- Understanding of AI systems and AI Testing concepts.
- Knowledge of Large Language Models (LLMs), RAG, and Chatbot AI systems.
- Experience designing and testing Generative AI systems and applications.




- Knowledge of LLM evaluation techniques and frameworks such as DeepEval and RAGAS.
- Ability to assess the output quality, coherence, and accuracy of Generative AI models.

Database Search Technologies:

- Solid understanding of SQL for testing databases (including joins, inserts, and updates).
- Understanding of Vector Databases such as OpenSearch or similar vector DB technologies.

Testing Tools:

- Experience with JMeter for performance and load testing.
- Familiarity with CI/CD tools like Jenkins or Bamboo.

Agile Methodology:

- Experience working in an Agile (Scrum) environment.
- Familiar with JIRA and Confluence for issue tracking and project management.

Web API Testing:

- Experience testing web applications and web services.

Product Performance:

- Capable of evaluating and troubleshooting performance-related issues.

Nice to Have:

Agentic AI Observability:

- Understanding of Agents, MCP, and Agentic AI systems.
- Familiarity with observability tools such as Langfuse.

Cloud Experience:

- Familiarity with AWS services such as AWS Bedrock for cloud-based AI environments.

Development Experience:

- A background in software development is a plus.

Industry Knowledge:

- Understanding of the Property and Casualty (PC) insurance industry is beneficial.

Job Category: Gen AI

Job Type: Full Time

Job Location: Remote

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

📌 Gen AI QA Automation Engineer (Coimbatore)
🏢 Nallas
📍 Coimbatore