Experience 5 Years Experience Education Bachelor s Master s degree Work Location Chennai India Chennai Remote Hybrid Key Responsibilities Design develop and maintain automated testing frameworks for Gen AI applications and agentic systems Implement evaluation pipelines to measure and improve LLM reliability and response quality Develop and operationalize hallucination and bias detection systems Collaborate with LLM Engineers to integrate evaluation checkpoints into model deployment workflows Define and manage evaluation datasets and metrics for automated quality reporting Partner with Product and Governance teams to ensure responsible AI testing and validation practices Continuously monitor and improve AI performance reliability and trust metrics Required Technical Skills Generative AI Quality Engineering Design automated testing frameworks for generative AI and multi-agent systems Develop evaluation pipelines to assess LLM responses for accuracy coherence and factual grounding Implement hallucination detection mechanisms using reference comparison confidence scoring or retrieval checks Create reproducible test datasets and evaluation benchmarks for prompt-response validation Automation Frameworks Tools Develop and maintain Python-based QA frameworks integrated with CI CD systems Experience with testing tools such as pytest LangSmith DeepEval or TruLens Automate testing of API-driven AI systems RAG pipelines and multi-agent conversations Implement regression performance and stress testing for large-scale AI applications Evaluation Metrics Analysis Define and compute LLM evaluation metrics precision recall coherence diversity factual accuracy toxicity Build evaluation dashboards and reporting tools for continuous model monitoring Analyze model drift prompt sensitivity and performance degradation over time Model Safety Reliability Develop and integrate toxicity bias and compliance detection checks into QA pipelines Collaborate with security and governance teams to ensure AI safety and alignment standards Implement guardrail testing and policy compliance frameworks for generative systems Infrastructure Integration Integrate QA pipelines into CI CD workflows for LLM deployment pipelines Experience with cloud-native tools AWS Azure GCP or OCI for automated test execution Implement test orchestration using Docker Kubernetes or serverless workflows Familiarity with API mocking synthetic test data generation and version-controlled evaluation artifacts Required Skills Experience 5 years of experience in QA Automation AI Testing or ML Evaluation Engineering Strong programming background in Python with experience in PyTest FastAPI or Playwright Understanding of LLM architectures prompt-response mechanics and RAG systems Experience with AI evaluation frameworks such as DeepEval TruLens or LangSmith Familiarity with data validation model observability and pipeline testing Proven ability to design automated evaluation frameworks for AI and ML models APPLY Close Drivestream s Employee Benefits Remuneration Drivestream offers competitive pay and attracts a diverse community of skilled individuals We recognize the value of investing in our talent Medical Disability and Life Insurance We provide an array of coverage options including full medical full dental and vision plans employee life insurance LTD and STD coverage flexible spending account and employee accidental death and dismemberment Leave Benefits Drivestream s generous paid leave programs feature vacation paid time off PTO holiday leave and bereavement leave Professional Development Our training and development programs include traditional classroom training online courses including leadership communication and project planning development strategic planning and management programs and professional society membership incentive Work Life Programs Drivestream offers Work-Life Integration options that help individuals manage their personal and professional responsibilities Options Include work from home and telecommuting day care flexible spending accounts internal job transfer and career mobility and health and wellness programs Community Involvement Drivestream believes in supporting community and philanthropic activities that allow our employees to engage in outreach and educational programs Awards Programs Drivestream recognizes and rewards our staff through various annual awards programs Retirement Perks Drivestream offers complete 401 k plans and annual profit-sharing contribution

📌 Gen Ai Qa Engineer – Ai Consulting & Strategy (Tamil Nadu)
🏢 Drivestream
📍 Tamil Nadu