Job description

Role - Quality Analyst - AI/ML (Level 1)
Experience - 1 to 3 years
Location - Mumbai / Hyderabad (Work from Office )
IMMEDIATE JOINERS ONLY

Must-have skills -
Generative AI, LLMs, and Agentic AI Systems
AI Evaluation Frameworks (RAGAS, DeepEval, TruLens, LangSmith, Promptfoo, etc.)
Python and AI/ML Development Libraries
Prompt Engineering, Testing, and Optimization
Automated Evaluation Pipelines and Benchmarking Frameworks
AI Safety, Guardrails, Bias Testing, and Responsible AI Practices
REST APIs, JSON, Vector Databases, and Knowledge Retrieval Systems
Manual Testing of AI/GenAI Applications (Functional, Exploratory, UAT, Regression, E2E)
Agent Reasoning, Tool Calling, Workflow Execution, and Response Validation




Automation Testing using Python Frameworks

Positive to Have Skills-
A/B Testing
Human-in-the-Loop Evaluation
Red Teaming
Evaluation Reporting & Dashboarding
Exposure to RAG-based Applications

Top 3 Responsibilities -
Design, execute, and automate evaluation strategies for Agentic AI applications.
Develop evaluation datasets, test cases, and benchmark suites to measure agent performance.
Analyze outputs for hallucinations, bias, safety risks, and provide actionable recommendations.

Please apply if you are an immediate joiners

📌 Qa Analyst Mumbai (India)
🏢 Winjit Technologies
📍 India