Role : Sr. Data Engineer-Databricks

Description

We
are looking for an experienced
Senior Databricks Data Engineer
with strong hands-on expertise in
PySpark, Delta Lake, and cloud data
pipelines. The ideal candidate will have experience integrating SAP systems (SAP BDC / SAP Datasphere / SAP ECC / SAP S/4HANA) with Databricks and a solid engineering background.

Responsibilities
- Design, develop, and maintain scalable data pipelines on Databricks using PySpark and Delta Lake.
- Build and optimize ETL/ELT workflows integrating SAP systems with Databricks (using OData, SAP BTP, CDC tools, DB Connectors, or partner solutions).
- Ingest structured and unstructured data from multiple SAP and non-SAP sources into the lakehouse.
- Implement data modeling (Medallion architecture – Bronze/Silver/Gold).
- Optimize Databricks clusters, jobs, workflows, and performance tuning of PySpark workloads.
- Ensure data quality, governance, lineage, and compliance across all pipelines.
- Troubleshoot data issues, analyze performance bottlenecks, and ensure pipeline reliability.




- Prepare technical documentation, deployment scripts, and reusable frameworks.

Required Skills
- 6-12 years of qualified experience in data engineering.
- Strong hands-on experience with Databricks, PySpark, Delta Lake, and distributed computing.
- Experience integrating SAP systems with cloud data platforms or Databricks.
- Preferred: SAP BDC / SAP Datasphere / SAP ECC / S/4HANA data model knowledge
- Familiarity with SAP connectors, OData, CDS Views, CPI, or 3rd-party extraction tools
- Expertise in SQL, data modeling, ETL/ELT pipeline development.
- Familiarity with
AI/ML concepts
,
LLMs
,
Databricks Genie
and their integration with data platforms.
- Experience with at least one cloud platform (Azure / AWS / GCP).
- Understanding of CI/CD, DevOps, Git, Databricks Repos, and orchestration tools (Airflow/Azure Data Factory).
- Strong debugging, optimization, and performance-tuning skills.
- Excellent comm

📌 Senior Data Engineer (India)
🏢 Bristlecone
📍 India