Seeking a Lead Software Engineer – Automation Testing with 6–8 years of experience in designing, developing, and executing automation frameworks for AI/ML-driven applications.

The ideal candidate brings hands-on experience with GenAI Testing, LangSmith, PyTest, Agentic frameworks, and solid AI/ML evaluation expertise.
- Strong programming expertise in Python.
- Hands-on expertise with LangSmith, Agentic frameworks, PyTest, BDD/V&V; frameworks.

AI / GenAI Testing
- Design and execute V&V; strategies for classical ML and GenAI/ agentic systems.
- Define test plans, tests cases, and execute them
- Evaluate AI models for accuracy, performance, safety, robustness, hallucinations, bias, and correctness.
- For GenAI, rely on LangSmith for LLM observability, tracing, monitoring, and evaluation, for offline and oline evaluation
- For classical ML UC, ensure V&V; tests are done during the whole life cycle of the model (ML monitoring)





Test Automation
- Build scalable automation frameworks using PyTest integrated with CI/CD pipelines.
- Develop automated test suites for APIs, microservices, and AI pipelines.
- Perform sanity, integration, functional, regression, and UAT testing across QA, UAT, and Prod environments.
- Implement automation strategies: Keyword-driven, Data-driven, BDD, Hybrid frameworks.

· Good understanding of Data Science related metrics (for both classical ML and GenAI UC)Experience with AI/Chatbot products and testing AI applications.

· Experience with Git, GitHub Copilot, and CI/CD pipelines.

· Knowledge of Azure/AWS cloud environments.

· Familiarity with MySQL, PostgreSQL, Oracle, SQL Server, NoSQL databases.

📌 Gen AI Testing (Bengaluru)
🏢 TalentOla
📍 Bengaluru