Description
Title: Senior Technical Specialist - PBI Databricks
Location: Remote
Duration: 12+ Months
Education: Any Degree
Exp Level: 8+ Years

Job Responsibilities:
Design and build conformed dimensions and transactional fact tables for enterprise reporting.
Develop and maintain data pipelines on Azure Databricks using PySpark, Spark SQL, and Delta Lake.
Implement best practices in dimensional modeling, including:
Star schemas
Conformed dimensions
Surrogate and deterministic hash keys (e.g., SHA-256)
Transactional fact design
Translate existing SQL Server and stored procedure logic into productive Spark-based transformations.
Implement incremental data processing and change data capture (CDC) pipelines.
Ensure data reconciliation with source systems to meet accuracy and audit standards.
Collaborate closely with a senior Databricks lead to deliver core data assets within defined timelines.





Required Skills:
8+ years of experience in data engineering, with robust focus on Azure Databricks.
Proficiency in:
PySpark
Spark SQL
Delta Lake
Deep expertise in dimensional modelling and data warehousing concepts.
Experience with:
Building scalable production pipelines
Incremental data loads and CDC patterns
Performance optimization in distributed systems
Strong understanding of relational databases and ability to interpret complex SQL/stored procedures

Positive to have

Experience with:
Lakeflow / Spark Declarative Pipelines or dbt
Unity Catalog
Exposure to financial/General Ledger data models
Familiarity with JD Edwards systems, including:
F0902 (balances)
F0911 (journal details)

📌 Senior Technical Specialist Mumbai
🏢 Datavail Infotech
📍 Mumbai