Role Overview

We're looking for a Senior AI Engineer to design, build, and ship production-grade GenAI systems for our clients. You'll own solutions end to end: RAG systems, chatbots, and AI agents that are reliable, observable, and cost-productive in production. This role is about applied LLM engineering, not training models from scratch.

Key Responsibilities

- Design and build RAG pipelines end to end, covering document ingestion, chunking strategies, embeddings, hybrid search, re-ranking, and retrieval evaluation

- Build conversational AI and chatbots with multi-turn memory, grounding, fallback handling, and human-handoff flows

- Develop AI agents and multi-agent workflows using frameworks such as LangGraph, LangChain, OpenAI Agents SDK, or CrewAI, including tool calling, planning, and state management

- Build and integrate MCP (Model Context Protocol) servers and clients to connect LLMs with client systems, APIs, and data sources

- Apply context engineering to decide what goes into the context window (retrieval, memory, tool outputs, summarisation, compaction) for accuracy and efficiency

- Optimise prompts systematically through versioning, A/B testing, and automated approaches (e.g., DSPy), not trial and error

- Manage tokenomics: track and reduce token usage and cost through prompt caching, model routing, batching, and right-sizing models per task

- Evaluate, optimise, and monitor AI system performance for accuracy, reliability, latency, and cost, using tracing and observability tools like LangSmith, Langfuse, or Arize Phoenix; after deployment, catch issues such as data drift, prompt or model regressions, and degraded accuracy, and ship fixes

- Build evaluation frameworks with golden datasets, LLM-as-judge, regression evals, and online/offline metrics, so that quality is measured before and after release

- Implement guardrails for prompt-injection defence, PII handling, output validation, and hallucination controls

- Collaborate with product managers, backend engineers, frontend developers,



and clients to identify opportunities for AI-driven solutions, applying emerging AI technologies, frameworks, and best practices to real business problems; mentor junior AI engineers

- Fine-tune open-weight models (e.g., Llama, Mistral, Qwen, Gemma) using LoRA/QLoRA, SFT, or DPO when prompting and RAG aren't enough, and decide when fine-tuning is actually justified (an added advantage)

Required Skills & Qualifications

- 5+ years of software engineering experience, with 2+ years building and shipping LLM-based applications to production

- Strong Python skills (async, APIs, testing); FastAPI or similar

- Hands-on experience with LLM provider APIs (OpenAI, Anthropic, Gemini, Bedrock / Azure OpenAI), including tool calling and structured outputs

- Proven experience building RAG systems, and an understanding of why they fail (retrieval quality, chunking, context limits)

- Experience building AI agents with LangGraph or an equivalent framework, and hands-on experience building MCP (Model Context Protocol) servers and clients

- Working knowledge of vector databases (pgvector, Qdrant, Pinecone, Weaviate, or similar) and embedding models

- Practical experience with LLM evaluation and observability tooling

- Clear understanding of LLM cost and latency trade-offs and how to optimise them

- Experience deploying AI services on a cloud platform (AWS, GCP, or Azure) with Docker and CI/CD

Good to Have

- Hands-on fine-tuning experience (Hugging Face, Unsloth, Axolotl) and model serving (vLLM, Ollama, TGI)

- Experience with the A2A (Agent-to-Agent) protocol

- Familiarity with prompt optimisation frameworks such as DSPy

- Multimodal experience (vision + language, document parsing)

- Exposure to AI security and governance practices (red-teaming, OWASP Top 10 for LLMs)

- Client-facing delivery experience in a services or consulting setup

- Open-source contributions in the GenAI space

What We Offer

- Opportunity to work on cutting-edge AI projects with chance to shape a growing AI practice: its patterns, tooling, and standards
- A collaborative, engineering-driven culture

- Competitive compensation and growth opportunities

📌 Senior AI Engineer (Coimbatore)
🏢 Mallow Technologies Private
📍 Coimbatore