Our customer's product is an AI-powered platform that helps businesses make better decisions and work more efficiently. It uses advanced analytics and machine learning to analyze large amounts of data and provide helpful insights and predictions. The platform is widely used in various industries, including healthcare, to optimize processes, improve customer experiences, and support innovation. It integrates easily with existing systems, making it easier for teams to make quick, data-driven decisions to deliver cutting-edge solutions.

Requirements
6+ years of hands-on experience in Data Engineering.
Solid proficiency in Python and advanced SQL, including query optimization, data modeling, and performance tuning.
Deep understanding of distributed data processing frameworks, particularly Apache Spark.
Robust practical experience with Apache Airflow for workflow orchestration and pipeline management.
Working knowledge of backend application development frameworks such as FastAPI.
Foundational understanding of LLMs, AI/GenAI concepts, and their practical applications within data platforms.
Familiarity with modern AI and data infrastructure concepts, including:
Vector Databases
Semantic Search
Knowledge Graphs
Retrieval-Augmented Generation (RAG)



architectures (preferred)
Hands-on experience with at least one major cloud platform: AWS, Azure, or GCP.
Hands-on experience with Git and CI/CD pipelines.

Responsibilities
Design and build highly reliable and scalable data pipelines using PySpark and big data technologies.
Collaborate with the data science team to develop new features that enhance model accuracy and performance.
Create standardized data models to improve consistency across various deployments.
Troubleshoot and resolve issues in existing ETL pipelines and optimize workflows.
Conduct POCs to evaluate current technologies and integrate additional data sources.
Follow and promote best practices for software development, ensuring high-quality solutions that meet requirements and deadlines.
Document development updates and maintain transparent technical documentation.

Benefits
Employment-based cooperation
Comprehensive insurance for you and your family (health, life, and accident)
Paid PTO policy (vacation, sick leaves, and public holidays)
Tech equipment provided
Udemy courses, workshops, trainings & expert knowledge-sharing
Flexible hours & work setup

📌 Senior Data Engineer Python, Llm, Rag, Vector Db Bengaluru
🏢 Intellectsoft
📍 Bengaluru