About the Company

AI Tester – Observability & Monitoring

About the Role

We are seeking experienced QA professionals with 5-7 years of testing/validation experience and exposure to AI/ML systems, GenAI/LLM testing, observability platforms, API testing, and data validation.

Responsibilities

• Testing and validation of AI/ML systems, GenAI/LLM testing, and model evaluation
• Working with AI observability and monitoring platforms (e.g., Langfuse)
• Performing API testing, backend validation, and SQL/data validation
• Monitoring model performance, drift detection, and alert validation
• Conducting functional and non-functional testing of AI applications
• Using test management tools (QTest, ALM, etc.)
• Applying Python-based test automation concepts
• Supporting AI governance, explainability, compliance, and regulated environments

Qualifications

• Experience: 5-7 Years
• Location:



Offshore

Required Skills

• AI/ML systems, GenAI/LLM testing, and model evaluation
• AI observability and monitoring platforms (e.g., Langfuse)
• API testing, backend validation, and SQL/data validation
• Model performance monitoring, drift detection, and alert validation
• Functional and non-functional testing of AI applications
• Test management tools (QTest, ALM, etc.)
• Python-based test automation concepts
• AI governance, explainability, compliance, and regulated environments

Preferred Skills

• Experience with a variety of observability and monitoring tools beyond Langfuse

Pay range and compensation package

Not specified.

Equal Prospect Statement

Not specified.

📌 AI Tester- Observability (India)
🏢 CosMicIT
📍 India