Title: Senior Technical Specialist - PBI Databricks

Location: Remote

Duration: 12+ Months

Education: Any Degree

Exp Level: 8+ Years

Job Responsibilities:
Design and build conformed dimensions and transactional fact tables for enterprise reporting.
Develop and maintain data pipelines on A zure Databricks using PySpark, Spark SQL, and Delta Lake.
Implement best practices in dimensional modeling, including:
Star schemas
Conformed dimensions
Surrogate and deterministic hash keys (e.g., SHA-256)
Transactional fact design
Translate existing SQL Server and stored procedure logic into productive Spark-based transformations.
Implement incremental data processing and change data capture (CDC) pipelines.
Ensure data reconciliation with source systems to meet accuracy and audit standards.
Collaborate closely with a senior Databricks lead to deliver core data assets within defined timelines.

Required Skills:




8+ years of experience in data engineering, with solid focus on Azure Databricks .
Proficiency in
PySpark
Spark SQL
Delta Lake
Deep expertise in dimensional modelling and data warehousing concepts.
Experience with
Building scalable production pipelines
Incremental data loads and CDC patterns
Performance optimization in distributed systems
Strong understanding of relational databases and ability to interpret complex SQL/stored procedures

Valuable to have

Experience with:
Lakeflow / Spark Declarative Pipelines or dbt
Unity Catalog
Exposure to financial/General Ledger data models
Familiarity with JD Edwards systems, including:
F0902 (balances)
F0911 (journal details)

📌 Data Engineer Bengaluru
🏢 Datavail
📍 Bengaluru