Job Description
Lead with Purpose. Partner with Impact.
nWe are seeking a seasoned Databricks Data Engineer with expertise in Azure cloud services and the Databricks Lakehouse platform. The role involves designing and optimizing large-scale data pipelines, modernizing cloud-based data ecosystems, and enabling secure, governed data solutions. Robust skills in SQL, Python, PySpark, ETL/ELT frameworks, and experience with Delta Lake, Unity Catalog, and CI/CD automation are essential.
nWhat youll Do:
n
- n
- Design, build, and optimize large-scale data pipelines on the Databricks Lakehouse platform, ensuring reliability, scalability, and governance. n
- Modernize the Azure-based data ecosystem, contributing to cloud architecture, distributed data engineering, data modeling, security, and CI/CD automation. n
- Utilize Apache Airflow and similar tools for orchestration and workflow automation. n




- Work with financial or regulated datasets, applying strong compliance and governance practices. n
- Develop and optimize ETL/ELT pipelines using Python, PySpark, Spark SQL, and Databricks notebooks. n
- Design and optimize Delta Lake data models for reliability, performance, and scalability. n
- Implement and manage Unity Catalog for RBAC, lineage, governance, and secure data sharing. n
- Build reusable frameworks using Databricks Workflows, Repos, and Delta Live Tables. n
- Create scalable ingestion pipelines for APIs, databases, files, streaming sources, and MDM systems. n
- Automate API ingestion and workflows using Python and REST APIs. n
- Support data governance, lineage, cataloging, and metadata initiatives n

📌 Lead - Data Engineer (Bengaluru)
🏢 Kestra
📍 Bengaluru