17 Nov
|
Zorba AI
|
Bengaluru
17 Nov
Zorba AI
Bengaluru
Role Overview We are looking for an experienced Data Engineering – AI professional with strong expertise in Python, SQL, Spark/PySpark, and large-scale data engineering. The ideal candidate should have hands-on experience designing and developing end-to-end data pipelines, distributed data processing solutions, and cloud-based data platforms. The candidate should also have exposure to AI-enabled data platforms, including LLM integration, Retrieval-Augmented Generation (RAG), vector databases, and AI application enablement. Prior experience in the Banking, Financial Services, or Insurance (BFSI) domain is mandatory.
Key Responsibilities
• Design, develop, and maintain scalable end-to-end data pipelines using Python, SQL, and Spark/PySpark.
• Build and optimize distributed data processing frameworks for large-scale structured and unstructured datasets.
• Develop robust data ingestion, transformation, integration, and processing workflows.
• Work with cloud platforms such as AWS, Azure, or GCP.
• Design and implement data solutions using Databricks, Snowflake, and modern Lakehouse architectures.
• Integrate LLM-based capabilities into enterprise data platforms and applications.
• Implement RAG-based solutions and work with vector databases for efficient retrieval and AI application enablement.
• Support MLOps/LLMOps practices, including deployment, CI/CD, orchestration, monitoring, and performance optimization.
• Implement data governance, data quality, security, and compliance standards.
• Collaborate with data scientists, AI engineers, architects, and business stakeholders to deliver scalable data and AI solutions. Mandatory Technical Skills
• Strong programming experience in Python and SQL.
• Hands-on experience with Apache Spark / PySpark.
• Strong Data Engineering experience, including end-to-end data pipelines and distributed processing.
• Experience with at least one cloud platform: AWS, Azure, or GCP.
• Experience with Databricks and/or Snowflake.
• Understanding of Lakehouse architecture and modern data platforms.
• Exposure to LLM integration, RAG, and vector databases.
• Knowledge of MLOps/LLMOps, CI/CD, orchestration, and monitoring.
• Understanding of data governance and security practices. Domain Requirement
• BFSI experience is mandatory.
• Candidates should have worked on data engineering solutions within Banking, Financial Services, or Insurance environments. Candidate Requirements
• 6–10 years of relevant experience in Data Engineering.
• Strong hands-on technical and problem-solving skills.
• Experience building scalable, reliable, and high-performance data platforms.
• Ability to work in a collaborative, fast-paced setting.
• Immediate to 30 days' notice period preferred.
Skills:
data engineering,python,ai
📌 Data Engineering – AI (Bengaluru)
🏢 Zorba AI
📍 Bengaluru