18 Nov
|
Zorba AI
|
Bengaluru
18 Nov
Zorba AI
Bengaluru
Role Overview
We are looking for an experienced Data Engineering – AI professional with strong expertise in Python, SQL, Spark/PySpark, and large-scale data engineering. The ideal candidate should have hands-on experience designing and developing end-to-end data pipelines, distributed data processing solutions, and cloud-based data platforms.
The candidate should also have exposure to AI-enabled data platforms, including LLM integration, Retrieval-Augmented Generation (RAG), vector databases, and AI application enablement. Prior experience in the Banking, Financial Services, or Insurance (BFSI) domain is mandatory.
Key Responsibilities
- Design, develop, and maintain scalable end-to-end data pipelines using Python, SQL, and Spark/PySpark.
- Build and optimize distributed data processing frameworks for large-scale structured and unstructured datasets.
- Develop robust data ingestion, transformation, integration, and processing workflows.
- Work with cloud platforms such as AWS, Azure, or GCP.
- Design and implement data solutions using Databricks, Snowflake, and modern Lakehouse architectures.
- Integrate LLM-based capabilities into enterprise data platforms and applications.
- Implement RAG-based solutions and work with vector databases for efficient retrieval and AI application enablement.
- Support MLOps/LLMOps practices, including deployment, CI/CD, orchestration, monitoring, and performance optimization.
- Implement data governance, data quality, security, and compliance standards.
- Collaborate with data scientists, AI engineers, architects, and business stakeholders to deliver scalable data and AI solutions.
Mandatory Technical Skills
- Strong programming experience in Python and SQL.
- Hands-on experience with Apache Spark / PySpark.
- Strong Data Engineering experience, including end-to-end data pipelines and distributed processing.
- Experience with at least one cloud platform: AWS, Azure, or GCP.
- Experience with Databricks and/or Snowflake.
- Understanding of Lakehouse architecture and contemporary data platforms.
- Exposure to LLM integration, RAG, and vector databases.
- Knowledge of MLOps/LLMOps, CI/CD, orchestration, and monitoring.
- Understanding of data governance and security practices.
Domain Requirement
- BFSI experience is mandatory.
- Candidates should have worked on data engineering solutions within Banking, Financial Services, or Insurance environments.
Candidate Requirements
- 6–10 years of relevant experience in Data Engineering.
- Strong hands-on technical and problem-solving skills.
- Experience building scalable, reliable, and high-performance data platforms.
- Ability to work in a collaborative, fast-paced environment.
- Immediate to 30 days' notice period preferred.
Skills: data engineering,python,ai
📌 Data Engineering – AI (DE AI) (Bengaluru)
🏢 Zorba AI
📍 Bengaluru