Role -Quality Analyst - AI/ML (Level 1)

Experience - 1 to 3 years

Location - Mumbai / Hyderabad (Work from Office )

IMMEDIATE JOINERS ONLY

Must-have skills -

Generative AI, LLMs, and Agentic AI Systems

AI Evaluation Frameworks (RAGAS, DeepEval, TruLens, LangSmith, Promptfoo, etc.)

Python and AI/ML Development Libraries

Prompt Engineering, Testing, and Optimization

Automated Evaluation Pipelines and Benchmarking Frameworks

AI Safety, Guardrails, Bias Testing, and Responsible AI Practices

REST APIs, JSON, Vector Databases, and Knowledge Retrieval Systems

Manual Testing of AI/GenAI Applications (Functional, Exploratory, UAT, Regression, E2E)

Agent Reasoning, Tool Calling, Workflow Execution, and Response Validation





Automation Testing using Python Frameworks

Positive to Have Skills-A/B Testing

Human-in-the-Loop Evaluation

Red Teaming

Evaluation Reporting & Dashboarding

Exposure to RAG-based Applications

Top 3 Responsibilities -1. Design, execute, and automate evaluation strategies for Agentic AI applications.

- Develop evaluation datasets, test cases, and benchmark suites to measure agent performance.
- Analyze outputs for hallucinations, bias, safety risks, and provide actionable recommendations.

Please apply if you are an immediate joiners

📌 QA Analyst (Hyderabad)
🏢 Winjit Technologies
📍 Hyderabad