Role & responsibilities
Design, build, and optimize enterprise Generative AI applications using Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG).
Validate AI-generated responses against business data sources (e.g., Looker Explore/semantic models) to ensure factual accuracy and semantic alignment.
Develop and optimize prompt engineering strategies to improve response quality, consistency, and reduce hallucinations.
Build and enhance RAG pipelines using vector databases, embeddings, semantic search, and retrieval optimization techniques.
Implement AI evaluation frameworks to measure response accuracy, relevance, faithfulness, latency, and overall LLM performance.
Collaborate with data engineering, analytics, and business teams to integrate AI capabilities into enterprise applications.
Develop scalable backend services and APIs using Python and FastAPI for AI solutions.
Monitor production AI systems using observability tools, perform root cause analysis, and continuously improve model performance.
Ensure AI solutions follow Responsible AI, security, governance, and enterprise best practices.

Preferred candidate profile




610+ years of overall software development experience with 2–4+ years in Generative AI/LLM engineering.
Strong hands-on expertise in Python and backend API development (FastAPI preferred).
Experience building production-grade LLM, RAG, and Agentic AI applications.
Hands-on experience with LangChain/LangGraph or similar orchestration frameworks.
Experience working with Vector Databases such as Azure AI Search, Pinecone, ChromaDB, FAISS, or OpenSearch.
Solid understanding of Prompt Engineering, embeddings, semantic search, hybrid retrieval, and reranking techniques.
Experience implementing AI Evaluation/Validation frameworks (RAGAS, DeepEval, LLM-as-a-Judge, hallucination detection, response quality evaluation).
Familiarity with Azure OpenAI, AWS Bedrock, OpenAI APIs, or similar enterprise AI platforms.
Solid knowledge of SQL and enterprise data models.
Experience with Docker, Git, CI/CD, and cloud deployment is preferred.
Excellent analytical, debugging, stakeholder communication, and problem-solving skills.
Experience working in enterprise, banking, financial services, or large-scale production settings is an added advantage.

📌 Ai Analyst Pune
🏢 Allegis Group
📍 Pune