We are looking for an experienced AI / Data Engineer to design and build scalable data pipelines and AI-powered applications. The ideal candidate will combine strong data engineering fundamentals with hands-on experience in LLM frameworks, Retrieval-Augmented Generation (RAG), and API development to deliver production-grade AI and data solutions.
Key Responsibilities :
- Design, develop, and optimize scalable data pipelines using Azure/AWS/GCP Databricks and PySpark.
- Develop AI-powered applications using LLMs, LangChain, and LlamaIndex.
- Design and implement RAG (Retrieval-Augmented Generation) solutions using enterprise knowledge bases.
- Develop REST APIs using Python (FastAPI/Flask) for AI and data services.
- Collaborate with Data Scientists, ML Engineers, and Business stakeholders to deliver AI-driven solutions.
- Optimize model performance, prompt engineering, and inference pipelines.
- Follow DevOps and MLOps best practices for deployment, monitoring, and maintenance.
Required Skills & Qualifications :
- 3+ years of experience in data engineering, with strong expertise in AWS/Azure/GCP Databricks and PySpark/Python.
- Proven experience building and maintaining ETL/ELT pipelines at scale.
- Hands-on experience with LLM frameworks such as LangChain and LlamaIndex.
- Practical knowledge of RAG architecture and enterprise knowledge base integration.
- Robust Python skills, including API development using FastAPI or Flask.
- Familiarity with prompt engineering and inference pipeline optimization.
- Understanding of DevOps/MLOps practices (CI/CD, model deployment, monitoring).
- Solid communication and collaboration skills to work across cross-functional teams.