Agentic AI Engineer (5-10 years of qualified experience)

Role Description

Roles & Responsibilities
- Implement AI/ML and Generative AI solutions using Python, with an emphasis on quality, reliability, efficiency, and scalability.
- Develop orchestration workflows that connect LLMs with enterprise APIs, databases, retrieval systems, and other tools.
- Design and optimize prompts, system instructions, tool descriptions, routing logic, and context-management strategies.
- Build RAG-based solutions, including document processing, chunking, embedding, retrieval, reranking, grounding, and context optimization.
- Define and implement evaluation frameworks for agentic AI systems, covering task completion, answer correctness, faithfulness, tool-selection accuracy, trajectory quality, latency, cost, safety, and reliability.
- Create representative evaluation datasets, test scenarios, error taxonomies, regression suites, and human-evaluation workflows.
- Partner with data scientists to evaluate model choices, prompt strategies, retrieval approaches,



and fine-tuning requirements where applicable.
- Work with data engineers to develop data pipelines, preprocessing utilities, data-quality checks, and feature or context engineering workflows.
- Integrate observability for agent execution, including prompts, responses, tool calls, intermediate steps, errors, token consumption, latency, and quality metrics.
- Use cloud platforms, preferably Azure or AWS, to deploy and scale AI models, agents, and supporting services.
- Establish engineering best practices for modularity, testing, reproducibility, versioning, documentation, security, and model governance.
- Troubleshoot and optimize agent behavior, model performance, retrieval quality, response latency, token usage, and infrastructure cost.
- Engage with business and technology stakeholders to gather requirements and translate them into measurable AI objectives and technical solutions.
- Conduct design and code reviews and provi

📌 Agentic Ai Engineer (Bengaluru)
🏢 State Street
📍 Bengaluru