21 Nov
|
Recognized
|
Bengaluru
21 Nov
Recognized
Bengaluru
Role Overview
We are looking for an experienced Data Engineering – AI professional with strong expertise in Python, SQL, Spark/PySpark, and large-scale data engineering. The ideal candidate should have hands-on experience designing and developing end-to-end data pipelines, distributed data processing solutions, and cloud-based data platforms.
The candidate should also have exposure to AI-enabled data platforms, including LLM integration, Retrieval-Augmented Generation (RAG), vector databases, and AI application enablement. Prior experience in the Banking, Financial Services, or Insurance (BFSI) domain is mandatory.
Key Responsibilities
-
Design, develop, and maintain scalable end-to-end data pipelines using Python, SQL, and Spark/PySpark.
-
Build and optimize distributed data processing frameworks for large-scale structured and unstructured datasets.
-
Develop robust data ingestion, transformation, integration, and processing workflows.
-
Work with cloud platforms such as AWS, Azure, or GCP.
-
Design and implement data solutions using Databricks, Snowflake, and contemporary Lakehouse architectures.
-
Integrate LLM-based capabilities into enterprise data platforms and applications.
-
Implement RAG-based solutions and work with vector databases for efficient retrieval and AI application enablement.
-
Support MLOps/LLMOps practices, including deployment, CI/CD, orchestration, monitoring, and performance optimization.
-
Implement data governance, data quality, security, and compliance standards.
-
Collaborate with data scientists, AI engineers, architects, and business stakeholders to deliver scalable data and AI solutions.
Mandatory Technical Skills
-
Strong programming experience in Python and SQL.
-
Hands-on experience with Apache Spark / PySpark.
-
Strong Data Engineering experience, including end-to-end data pipelines and distributed processing.
-
Experience with at least one cloud platform: AWS, Azure, or GCP.
-
Experience with Databricks and/or Snowflake.
-
Understanding of Lakehouse architecture and modern data platforms.
-
Exposure to LLM integration, RAG, and vector databases.
-
Knowledge of MLOps/LLMOps, CI/CD, orchestration, and monitoring.
-
Understanding of data governance and security practices.
Domain Requirement
-
BFSI experience is mandatory.
-
Candidates should have worked on data engineering solutions within Banking, Financial Services, or Insurance environments.
Candidate Requirements
-
6–10 years of relevant experience in Data Engineering.
-
Strong hands-on technical and problem-solving skills.
-
Experience building scalable, reliable, and high-performance data platforms.
-
Ability to work in a collaborative, fast-paced environment.
-
Immediate to 30 days' notice period preferred.
📌 Data Engineering AI (DE AI) (Bengaluru)
🏢 Recognized
📍 Bengaluru