About US:

Straive is a global leader in data analytics and AI operationalization, helping enterprises embed advanced AI and data capabilities into core business workflows to deliver measurable business outcomes and ROI. With a workforce of ~20,000 professionals serving 350+ clients across 30+ markets, Straive combines technical scale with deep domain expertise. A key differentiator is its network of over 6,000 subject matter experts who specialize in managing and enriching complex, unstructured data enabling organizations to build AI systems grounded in accuracy, context, and business relevance.

The company continues to earn recognition from leading industry analysts and was recently named a Leader in AIM’s 2026 Generative AI and Data Engineering PeMa Quadrants. Backed by EQT a purpose-driven global investment organization which was ranked among the world’s leading private equity firms by PEI in 2025 Straive is positioned as a high-value alternative to traditional IT services providers, combining domain-led intelligence with AI execution at scale.

Role & responsibilities

•Design and execute comprehensive test strategies tailored for AI/ML models, LLM-based applications, and data pipelines to ensure system accuracy and reliability.

•Develop and maintain automated test frameworks for model validation, regression testing,

and performance benchmarking using Pytest.

- Evaluate LLM and RAG outputs for structural correctness, factual consistency, relevance,

hallucination risks, and demographic bias.

•Build, update, and manage robust evaluation datasets, golden/ground truth sets, and adversarial edge-case test suites.

•Monitor production models to track performance metrics, identify data/model drift,



and catch anomalous behaviors over time.

•Validate incoming data streams, data pipelines, and feature stores to confirm completeness and accuracy of training/inference data.

•Collaborate directly with Data Scientists and Machine Learning Engineers to define explicit acceptance criteria, quality thresholds, and prompt regression standards.

•Conduct ethical and compliance assessments, including bias audits and model explainability checks.

•Document distinct defects, edge cases, and failure patterns specific to non-deterministic AI behaviors.

Preferred profile

Experience: 3 to 6 years of core QA experience, with a mandatory 1 to 2 years specialized in AI/ML quality assurance and LLM testing.

Education: Bachelor's or Master's degree in Computer Science, Engineering, or a closely related technical field.

Technical Expertise: Robust hands-on coding in Python for data analysis and test automation.

Tools Knowledge: Proven familiarity with LLM evaluation frameworks (RAGAS, DeepEval,

Promptfoo, LangSmith) alongside traditional QA tools (Pytest, Selenium, Postman).

Data Skills: Working knowledge of data quality frameworks such as Great Expectations or dbt test structures.

Domain Context: Solid understanding of the Machine Learning lifecycle (training, validation,

deployment, monitoring) and RAG system architectures.

Job Location - Bangalore or Mumbai

Work mode- Hybrid

Straive is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive setting for all employees. All employment decisions are based on business needs, job requirements, and individual qualifications, without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, veteran, or disability status."

📌 Quality Assurance Engineer (Bengaluru)
🏢 Straive
📍 Bengaluru