Role : Sr. Data Engineer-Databricks
Description
We are looking for an experienced Senior Databricks Data Engineer with strong hands-on expertise in Py Spark, Delta Lake, and cloud data pipelines. The ideal candidate will have experience integrating SAP systems (SAP BDC / SAP Datasphere / SAP ECC / SAP S/4 HANA) with Databricks and a solid engineering background.
Responsibilities
- Design, develop, and maintain scalable data pipelines on Databricks using Py Spark and Delta Lake.
- Build and optimize ETL/ELT workflows integrating SAP systems with Databricks (using OData, SAP BTP, CDC tools, DB Connectors, or partner solutions).
- Collaborate with AI teams to integrate LLMs and advanced ML models into data workflows.
- Ingest structured and unstructured data from multiple SAP and non-SAP sources into the lakehouse.
- Implement data modeling (Medallion architecture – Bronze/Silver/Gold).
- Optimize Databricks clusters, jobs, workflows, and performance tuning of Py Spark workloads.
- Ensure data quality, governance, lineage, and compliance across all pipelines.




- Troubleshoot data issues, analyze performance bottlenecks, and ensure pipeline reliability.
- Prepare technical documentation, deployment scripts, and reusable frameworks.
Required Skills
- 8-12 years of skilled experience in data engineering.
- Strong hands-on experience with Databricks, Py Spark, Delta Lake, and distributed computing.
- Experience integrating SAP systems with cloud data platforms or Databricks.
- Preferred: SAP BDC / SAP Datasphere / SAP ECC / S/4 HANA data model knowledge
- Familiarity with SAP connectors, OData, CDS Views, CPI, or 3rd-party extraction tools
- Expertise in SQL, data modeling, ETL/ELT pipeline development.
- Familiarity with AI/ML concepts, LLMs, Databricks Genie and their integration with data platforms.
- Experience with at least one cloud platform (Azure / AWS / GCP).
- Understanding of CI/CD, Dev Ops, Git, Databricks Repos, and orchestration tools (Airflow/Azure Data

📌 Senior data engineer (Bengaluru)
🏢 Bristlecone
📍 Bengaluru