Role & responsibilities

- Design, build, and optimize enterprise Generative AI applications using Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG).
- Validate AI-generated responses against business data sources (e.g., Looker Explore/semantic models) to ensure factual accuracy and semantic alignment.
- Develop and optimize prompt engineering strategies to improve response quality, consistency, and reduce hallucinations.
- Build and enhance RAG pipelines using vector databases, embeddings, semantic search, and retrieval optimization techniques.
- Implement AI evaluation frameworks to measure response accuracy, relevance, faithfulness, latency, and overall LLM performance.
- Collaborate with data engineering, analytics, and business teams to integrate AI capabilities into enterprise applications.
- Develop scalable backend services and APIs using Python and FastAPI for AI solutions.
- Monitor production AI systems using observability tools, perform root cause analysis, and continuously improve model performance.
- Ensure AI solutions follow Responsible AI, security, governance, and enterprise best practices.

Preferred candidate profile





- 610+ years of overall software development experience with 2–4+ years in Generative AI/LLM engineering.
- Strong hands-on expertise in Python and backend API development (FastAPI preferred).
- Experience building production-grade LLM, RAG, and Agentic AI applications.
- Hands-on experience with LangChain/LangGraph or similar orchestration frameworks.
- Experience working with Vector Databases such as Azure AI Search, Pinecone, ChromaDB, FAISS, or OpenSearch.
- Strong understanding of Prompt Engineering, embeddings, semantic search, hybrid retrieval, and reranking techniques.
- Experience implementing AI Evaluation/Validation frameworks (RAGAS, DeepEval, LLM-as-a-Judge, hallucination detection, response quality evaluation).
- Familiarity with Azure OpenAI, AWS Bedrock, OpenAI APIs, or similar enterprise AI platforms.
- Solid knowledge of SQL and enterprise data models.
- Experience with Docker, Git, CI/CD, and cloud deployment is preferred.
- Excellent analytical, debugging, stakeholder communication, and problem-solving skills.
- Experience working in enterprise, banking, financial services, or large-scale production environments is an added advantage.

📌 AI Analyst (Pune)
🏢 Allegis Group
📍 Pune