Primary Skillsets
Automation Testing with 8-12 years of experience in designing developing and executing automation frameworks for AIML driven applications. The ideal candidate brings handson experience with GenAI Testing LangSmith, PyTest Agentic frameworks and robust AIML evaluation expertise.
Detailed JD
Role Overview
We are seeking a Lead Software Engineer Automation Testing with 8-12 years of experience in designing developing and executing automation frameworks for AIMLdriven applications. The ideal candidate brings handson experience with GenAI Testing, LangSmith, PyTest Agentic frameworks and strong AIML evaluation expertise.
Key Responsibilities
AI GenAI Testing
Design and execute VV strategies for classical ML and GenAI agentic systems.
Define test plans tests cases and execute them.
Evaluate AI models for accuracy performance safety robustness hallucinations bias and correctness.
For GenAI rely on LangSmith for LLM observability tracing monitoring and evaluation for offline and online evaluation.
For classical ML UC ensure VV tests are done during the whole life cycle of the model ML monitoring
Test Automation
Build scalable automation frameworks using PyTest integrated with CICD pipelines
Develop automated test suites for APIs microservices and AI pipelines
Perform sanity integration functional regression and UAT testing across QA UAT and Prod environments
Implement automation strategies Keyword driven, Datadriven BDD Hybrid frameworks
Quality Assurance Reporting
Analyze test results and provide insights to engineering teams
Log and track defects using standard bug reporting tools
Maintain high quality documentation for all test processes and reports
Disclaimer : This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.