Required Skills
AI/ML Testing
PyTest
Test Automation
Python
LangSmith
CI/CD
API Testing
Cloud Platforms
SQL &
Databases

Nice to Have
Git
Data Science Metrics

Qualification | B.E or B.Tech Experience (in years) | 8 | We are seeking a Lead Software Engineer – Automation Testing with 6–8 years of experience in designing, developing, and executing automation frameworks for AI/ML-driven applications. The ideal candidate brings hands-on experience with GenAI Testing, LangSmith, PyTest, Agentic frameworks, and solid AI/ML evaluation expertise.

Key Responsibilities AI / GenAI Testing • Design and execute V&V; strategies for classical ML and GenAI/ agentic systems.
Define test plans, tests cases, and execute them • Evaluate AI models for accuracy, performance, safety, robustness, hallucinations, bias, and correctness.
For GenAI, rely on LangSmith for LLM observability, tracing, monitoring, and evaluation, for offline and oline evaluation • For classical ML UC, ensure V&V; tests are done during the whole life cycle of the model (ML monitoring) Test Automation • Build scalable automation frameworks using PyTest integrated with CI/CD pipelines.
Develop automated test suites for APIs, microservices, and AI pipelines.
Perform sanity, integration, functional, regression,



and UAT testing across QA, UAT, and Prod environments.
Implement automation strategies: Keyword-driven, Data-driven, BDD, Hybrid frameworks.

Quality

Assurance &
Reporting • Analyze test results and provide insights to engineering teams.
Log and track defects using standard bug-reporting tools.
Maintain high-quality documentation for all test processes and reports.

Skill Set

Required | • 6–8 years of experience in software testing & automation.
Strong programming expertise in Python.
Positive understanding of Data Science related metrics (for both classical ML and GenAI UC)Experience with AI/Chatbot products and testing AI applications.
Experience with Git, GitHub Copilot, and CI/CD pipelines.
Knowledge of Azure/AWS cloud environments.
Familiarity with MySQL, PostgreSQL, Oracle, SQL Server, NoSQL databases.
Solid verbal & written communication skills.
Experience with multi-stakeholder delivery models including onshore–offshore.
Qualification: B.E./B.Tech Soft Skills • Analytical &
Critical Thinking • Curiosity &
Continuous Learning • Strategic Communication(Active Listening, Technical Translations to business impacts)

📌 Lead Software Engineer Automation Testing M L Bengaluru
🏢 Cigres Technologies
📍 Bengaluru