Lead Data Engineer
Experience- 10 years
Location- Bangalore
Overview
We are seeking a highly experienced Lead Data Engineer to lead the design, development, and delivery of scalable, reliable, and cost-efficient data platforms. The ideal candidate will possess deep expertise in modern data engineering technologies, including distributed data processing, ETL/ELT pipeline development, data modeling, and workflow orchestration, along with hands-on experience with the Databricks Lakehouse Platform and its medallion (Bronze/Silver/Gold) architecture
. This role also requires a solid understanding of Large Language Models (LLMs) and Gen AI data ecosystems, enabling the development of high-quality datasets and retrieval-ready pipelines for AI-powered applications. As a technical leader, you will mentor engineering teams, establish best practices, and collaborate closely with architects, data scientists, and business stakeholders to deliver robust data solutions
.
Key Responsibilities
- Lead the architecture, development,



and optimization of scalable data platforms supporting both batch and streaming workloads.
- Design, implement, and maintain ETL/ELT pipelines for data ingestion, transformation, and curation using the Databricks Lakehouse Platform and medallion architecture.
- Define data models, schemas, and storage strategies for data lakes and data warehouses to support analytics, reporting, machine learning, and Gen AI initiatives.
- Develop and curate high-quality datasets, feature engineering pipelines, and retrieval-ready data stores, including embeddings and vector-based data structures, for LLM-powered applications.
- Establish engineering standards, coding best practices, code review processes, and CI/CD pipelines to ensure maintainable and reliable solutions.
- Build and automate end-to-end data workflows using orchestration tools such as Apache Airflow or equivalent platforms.
- Lead migrations from legacy on-premises or cloud-based data warehouses to up-to-date c

📌 Data engineer (Bengaluru)
🏢 Trantor
📍 Bengaluru