Hiring: AI Engineer – MCP Engineering & AI Evaluation

? Location: Bangalore (Hybrid)

? Work Mode: Hybrid

? Shift: CET Shift (12:00 PM – 8:00 PM IST) We are looking for an experienced AI Engineer with expertise in LLMs, AI Agents, MCP Servers, RAG, and AI Evaluation Frameworks to build and evaluate enterprise-grade AI solutions.

Key Responsibilities

Design, build, and maintain MCP (Model Context Protocol) servers exposing tools and data products to AI agents.

Develop evaluation frameworks and automated test suites for AI agents, connectors, and data products.

Curate evaluation datasets including golden datasets, edge cases, and SAP workflow-based scenarios.

Measure AI quality using metrics such as groundedness, hallucination rate, task success, and response accuracy.

Benchmark enterprise AI agents against external LLMs (Claude, Perplexity, Mistral, etc.).

Implement evaluation pipelines using DeepEval, RAGAS, or similar frameworks.

Assess AI tools and recommend build, buy, or integration strategies.

Explore deployment of local LLMs using Ollama and llama.cpp.

Collaborate with Product, Business AI, Security, and Engineering teams to deliver production-ready AI solutions.

Research emerging AI technologies and contribute to next-generation enterprise AI initiatives.





Required Skills

5+ years of software engineering experience with Python.

Strong experience in LLM applications, AI Agents, RAG, Prompt Engineering, Context Engineering, and Tool Calling.

Hands-on experience with MCP Servers or API/Agent integration frameworks.

Experience with AI evaluation frameworks such as DeepEval, RAGAS, or equivalent.

Knowledge of AI orchestration frameworks such as LangChain, LangGraph, CrewAI, AutoGen, or similar.

Experience with vector databases and embedding pipelines.

Familiarity with SAP AI Core, SAP AI Launchpad, SAP BTP, or cloud AI platforms (Azure AI, AWS AI, GCP Vertex AI).

Solid understanding of Kubernetes, Docker, Microservices, CI/CD, and cloud-native development.

Knowledge of Responsible AI, AI governance, observability, and enterprise security.

Nice to Have

Experience deploying local LLMs using Ollama, llama.cpp, GGUF, and model quantization.

Experience benchmarking AI agents and multi-agent systems.

Exposure to SAP enterprise workflows and AI-driven automation. ? Interested candidates can reach out via DM or share their updated resume at [email protected]. Immediate joiners are preferred.

📌 Artificial Intelligence Engineer (Bengaluru)
🏢 Runmap Infotech Private
📍 Bengaluru