We are looking for an AI Engineer to join our team and help test, evaluate, and improve advanced Agentic AI Systems.
? Key Responsibilities
- Test, evaluate, and improve the performance and quality of AI agents.
- Design and execute evaluation datasets, Q&A; sets, and multi-turn conversational scenarios.
- Define and measure agent evaluation metrics, including tool-calling performance, accuracy, reliability, and overall effectiveness.
- Benchmark agent performance and identify areas for improvement.
- Analyze large volumes of business documentation and data to quickly understand complex domains.
- Work on improving agent quality, reliability, and overall user outcomes.
? Required Skills
- 5+ years of experience in AI/ML, Generative AI, or related fields.
- Strong proficiency in Python with a solid understanding of APIs and system integrations.
- Robust knowledge of Generative AI, LLMs, and Agentic Systems.
- Experience with AI/LLM evaluation, benchmarking, and performance measurement.
- Strong analytical and problem-solving skills.
- Ability to work with complex datasets and translate business requirements into effective evaluation scenarios.
? Interested? Share your profile/resume with us or apply directly.
📌 AI ngineer (India)
🏢 BlueRose Technologies
📍 India
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.