Job Description

Roles & Responsibilities :
AI Quality Strategy:Develop and own the evaluation framework for GenAI solutions, focusing onFaithfulness, Relevancy, and Hallucination detectionusing LLM-as-a-judge frameworks.

- Hybrid Test Automation:Architect a dual-layered automation suite:

- Deterministic:E2E UI (Playwright) and API testing (Pytest/Requests).

- Probabilistic:Automated evaluation of non-deterministic LLM outputs.

- Shift-Left Integration:Embed automated quality checks directly intoGitHub Workflows, enabling seamless CI/CD.

Performance & Resilience:Lead JMeter-based performance testing.

Qualifications

Educational qualification:

- Experience:8+ years in Software QA

- Problem Solving:Ability to define quality in an ambiguous, non-deterministic AI landscape.

- Education:Bachelor's or Master's degree in Computer Science, Software Engineering, or a related field.

Experience :

- 8+ years in Software QA

Mandatory/requires Skills :
Automation & Tooling

- Python Mastery:Expert-level Python skills for building custom test tooling and automation scripts.

- Testing Stack:Hands-on proficiency withPytest(API),Playwright(E2E), andJMeter(Performance).

- DevOps:Advanced experience designing and maintainingGitHub Actions/Workflowsfor automated test execution.

Core AI & LLM Expertise





- Learning Agility in GenAI: High capability and interest in rapidly mastering AI evaluation concepts. You should be prepared to quickly upskill in automated metrics for LLMs (such as Faithfulness, Relevancy, and Groundedness).

- Exposure to LLM Logic: Basic familiarity with how LLMs function (e.g., prompting, context windows). You should be comfortable exploring and implementing LLM-as-a-Judge strategies, where high-reasoning models help grade application-specific outputs.

- Orientation toward RAG Systems: Interest in understanding the mechanics of Retrieval-Augmented Generation (RAG). You will be responsible for defining how we validate the accuracy of data retrieved from our engineering context catalogues and vector databases.

- Data-Driven Quality Mindset: A robust desire to move beyond binary Pass/Fail results toward probabilistic quality monitoring, utilizing tools like Langfuse to analyze live traces and performance trends.

Preferred Skills :

Additional Information

Why Join MiDAS

You won't just be testing software you will be defining the quality standards for the future ofAI-First Engineering. Your work will directly impact the speed and reliability of vehicle software development globally.

📌 Quality Assurance (QA) - Lead - MiDAS (India)
🏢 Robert Bosch
📍 India