Role summary

We’re looking for a hands-on Senior AI Engineer to design, build, and ship production-grade LLM applications, including RAG systems and tool-using agents. You’ll work with Product, Architecture, and Engineering to take AI use cases from proof of concept to reliable, cost-effective production solutions.

Responsibilities

- Design, build, and deploy production LLM applications, from prototype through monitoring and iteration.

- Take end-to-end ownership of assigned AI features, from technical approach through delivery and production support.

- Build and optimize RAG pipelines: ingestion, parsing, chunking, embeddings, indexing, retrieval, reranking, and grounded generation.

- Develop AI agents with tool calling, multi-step reasoning, and integration with external APIs and enterprise systems, including through MCP.

- Apply prompt and context engineering, structured outputs, and model selection to improve quality.

- Build evaluation frameworks to measure accuracy, retrieval relevance, groundedness, hallucination, and regressions.

- Optimize for latency, cost, scalability, and reliability.

- Develop backend services and REST APIs in Python (FastAPI or Flask).

- Communicate AI capabilities, limitations, and trade-offs clearly to technical and non-technical stakeholders, including clients; write design notes and documentation, and present demos.

- Mentor junior engineers through code reviews, pairing, and knowledge sharing, while remaining a hands-on individual contributor.

- Collaborate with Product and Architecture to translate business needs into scalable solutions, and contribute reusable components and engineering standards.

Required qualifications

- 7+ years of industry experience, including 2+ years building LLM-powered applications.

- Robust Python skills and experience building production backend services and APIs.

- Strong understanding of LLM fundamentals,



including transformer architecture and attention, tokenization and context windows, embeddings and semantic similarity, decoding parameters (temperature, top-p), causes of hallucination, and when to use prompting, RAG, or fine-tuning.

- Hands-on experience with RAG architectures and at least one vector database (pgvector, Pinecone, Qdrant, Weaviate, Chroma, or similar).

- Hands-on experience integrating LLMs into applications through APIs (OpenAI, Anthropic, Gemini, AWS Bedrock, Azure OpenAI) or open-weight models, including function calling and structured outputs.

- Experience building AI agents or tool-calling workflows.

- Hands-on experience with LangGraph (and familiarity with LlamaIndex or LangChain), including multi-agent design.

- Working knowledge of Model Context Protocol (MCP), including building or consuming MCP servers to connect agents to tools and data sources.

- Experience evaluating LLM applications (e.g., retrieval metrics, LLM-as-a-Judge, regression testing).

- Awareness of LLM security risks such as prompt injection and data leakage, and experience implementing guardrails and access controls for agents and tools.

- Working knowledge of AWS or an equivalent cloud (e.g., EC2, ECS, Lambda, S3), Docker, CI/CD, and production monitoring, including secure handling of sensitive data.

- Excellent written and verbal communication skills, with the ability to explain complex AI concepts and set realistic expectations on accuracy and behaviour.

- Ability to design, build, and debug end-to-end solutions independently.

Good to have

- Hybrid search, reranking, and query transformation.

- AI gateway architectures.

- Document intelligence: PDF processing, OCR, information extraction.

- AI observability, model routing, and PII protection.

- Background in ML/deep learning, and model fine-tuning (e.g., LoRA/QLoRA).

- Open-source model serving (Ollama, vLLM).

- Experience in regulated domains (healthcare, HRTech, fintech, insurance)

📌 Senior Artificial Intelligence Engineer (Indore)
🏢 Pepcus Software Services
📍 Indore