Must have: AI Testing, LLM, Python, automation testing

Your Key Responsibilities

- Define and own test strategy, planning, execution, and reporting for AI-based systems
- Conduct AI model validation across functional, performance, and statistical metrics
- Perform data validation activities including data collection, augmentation, exploratory analysis, bias detection, and privacy assessment
- Identify discrepancies, anomalies, and risks in model outputs and ensure timely resolution with development teams
- Lead defect management and collaborate with cross-functional teams for resolution
- Engage with client stakeholders to provide status updates and resolve issues
- Prepare test reports and support sign-off and closure activities
- Lead distributed teams across geographies and ensure delivery excellence





Skills and attributes for success

- Robust understanding of AI/ML concepts, LLMs, and Agentic AI systems
- Hands-on experience in AI testing techniques such as bias testing, drift testing, metamorphic testing, and back-to-back testing
- Expertise in test automation tools (e.g., Selenium, Playwright, PyTest)
- Proficiency in Python programming and AI evaluation frameworks
- Exposure to GitHub Copilot or similar AI-assisted development tools
- Good understanding of cloud platforms such as Azure, AWS, or GCP
- Strong analytical, problem-solving, and communication skills
- Experience working in Agile environments

📌 AI Testing (Kolkata)
🏢 EY
📍 Kolkata