Hiring: AI Engineer – MCP Engineering & AI Evaluation

? Location: Bangalore (Hybrid)

? Work Mode: Hybrid

? Shift: CET Shift (12:00 PM – 8:00 PM IST)

We are looking for an experienced AI Engineer with expertise in LLMs, AI Agents, MCP Servers, RAG, and AI Evaluation Frameworks to build and evaluate enterprise-grade AI solutions.

Key Responsibilities

- Design, build, and maintain MCP (Model Context Protocol) servers exposing tools and data products to AI agents.
- Develop evaluation frameworks and automated test suites for AI agents, connectors, and data products.
- Curate evaluation datasets including golden datasets, edge cases, and SAP workflow-based scenarios.
- Measure AI quality using metrics such as groundedness, hallucination rate, task success, and response accuracy.
- Benchmark enterprise AI agents against external LLMs (Claude, Perplexity, Mistral, etc.).
- Implement evaluation pipelines using DeepEval, RAGAS, or similar frameworks.
- Assess AI tools and recommend build, buy, or integration strategies.
- Explore deployment of local LLMs using Ollama and llama.cpp.
- Collaborate with Product, Business AI, Security, and Engineering teams to deliver production-ready AI solutions.
- Research emerging AI technologies and contribute to next-generation enterprise AI initiatives.





Required Skills

- 5 years of software engineering experience with Python.
- Solid experience in LLM applications, AI Agents, RAG, Prompt Engineering, Context Engineering, and Tool Calling.
- Hands-on experience with MCP Servers or API/Agent integration frameworks.
- Experience with AI evaluation frameworks such as DeepEval, RAGAS, or equivalent.
- Knowledge of AI orchestration frameworks such as LangChain, LangGraph, CrewAI, AutoGen, or similar.
- Experience with vector databases and embedding pipelines.
- Familiarity with SAP AI Core, SAP AI Launchpad, SAP BTP, or cloud AI platforms (Azure AI, AWS AI, GCP Vertex AI).
- Strong understanding of Kubernetes, Docker, Microservices, CI/CD, and cloud-native development.
- Knowledge of Responsible AI, AI governance, observability, and enterprise security.

Nice to Have

- Experience deploying local LLMs using Ollama, llama.cpp, GGUF, and model quantization.
- Experience benchmarking AI agents and multi-agent systems.
- Exposure to SAP enterprise workflows and AI-driven automation.

? Interested candidates can reach out via DM or share their updated resume at [email protected]. Immediate joiners are preferred.

📌 Artificial Intelligence Engineer (Bengaluru)
🏢 Runmap Infotech Private
📍 Bengaluru