- Step into a role where you help teams build Generative AI solutions that are not only powerful but also trustworthy safe and testable
- As part of a collaborative consulting environment you ll work closely with product engineering and risk stakeholders to design practical assurance approaches for LLM powered systems balancing innovation with reliability
- You ll contribute to real world AI validation efforts from defining test strategies and evaluation metrics to executing red teaming exercises and documenting outcomes that leadership can act on
- If you enjoy combining structured testing discipline with the rapid evolving world of GenAI this is a chance to shape how AI quality safety and responsibility are measured and improved while learning alongside motivated teams solving meaningful problems
Key Responsibilities:
- Define and execute testing strategies for Generative AI LLM systems covering functional non functional and safety focused validation
- Design evaluation frameworks and test suites using tools such as DeepEval to measure quality robustness and consistency of model outputs
- Plan and conduct red teaming exercises to identify vulnerabilities e
- g
- prompt injection jailbreaks harmful content data leakage and recommend mitigations
- Establish Responsible AI assurance checks aligned to fairness transparency privacy and safety expectations
- Create clear test plans evidence dashboards and reports that communicate risks findings and remediation priorities to stakeholders
- Collaborate with engineering and product teams to integrate AI testing into delivery pipelines and improve release readiness criteria
- Support incident triage and root cause analysis for model behavior issues regressions and evaluation drift across versions
Technical Requirements:
- BTech BE or equivalent technical degree
- 3 9 years of experience in testing quality engineering with hands on exposure to Generative AI or LLM testing initiatives
- Working knowledge of AI testing concepts including test design evaluation metrics and result interpretation for LLM outputs
- Experience contributing to structured documentation such as test plans test evidence and defect risk reporting
Additional Responsibilities:
- Proven experience executing red teaming for LLM applications and translating findings into actionable controls and mitigations
- Practical experience implementing Responsible AI practices and assurance workflows across the AI lifecycle
- Strong understanding of LLM failure modes hallucinations toxicity bias prompt sensitivity and methods to test and reduce them
- Experience building automated evaluation pipelines and regression suites for LLM systems using DeepEval or similar frameworks
- Consulting experience stakeholder management requirement discovery and delivering clear outcomes under ambiguity