Hiring at Cerebra IT Services | Contract to Hire(C2H)-MNC client
Role: CardioAI Testing Lead
Exp: 8 + years
Location: Bangalore, Pune, Noida, Kolkata
Skills
Lead end-to-end QA strategy covering functional| integration| UAT| regression| and performance testing across CardioAI platform components (APIs| agents| front-end| data systems).Prior Experience with Leading testing for AI based system Required.Own test planning| execution governance| defect management| and status reporting including risks| dependencies| and release readiness.Coordinate with cross-functional teams (Data| AIML| Salesforce| ServiceMax| APIs)
to ensure complete system validation and traceability.Drive automation strategy| testing standards| and AI testing tools selectionPrior experience leading testing initiatives for AI-based systems is mandatoryExtensive experience in testing RAG (Retrieval-Augmented Generation) systems| AI agents| chatbots| and conversational AI platforms.Ability to design and execute offline and online evaluation pipelines for LLM applications| ensuring continuous quality monitoring and model performance assessment.Solid understanding of AI evaluation metrics and principlesHands-on experience with AI evaluation and observability platforms such as RAGAS| Langfuse| LangSmith Other LLM evaluation and monitoring frameworks