Role & responsibilities

1. Build and maintain ingestion pipelines (CDC from CarePath, Recent Leaf daily loads, DEG SFTP feeds) into Bronze per defined patterns
2. Implement Bronze > Silver > Gold transformation logic in Synapse/Spark notebooks (or Fabric pipelines/notebooks), based on STTM specs authored by the Data SME/Modeler
3. Physically build the dimensional model (tables, keys, partitions) that the SME/Modeler designed and the Architect approved
4. Implement DQ rules and checks defined by the DQ Lead into pipeline code, execute & automate
5. Optimize Spark pool performance and job scheduling (addressing current cost/performance issues & migrate Fabric capacity-based (F-SKU) pricing as a potential cost-optimization path
6. Troubleshoot pipeline failures, job clashes, and legacy notebook code & support production stability
7. Implement MDM matching logic as executable pipeline jobs

Preferred candidate profile :

PySpark/Spark SQL, Azure Synapse pipelines, Microsoft Fabric (Data Factory, Lakehouse, Notebooks, OneLake), Delta Lake, Python, CI/CD for data pipelines, performance tuning, ADF/Synapse orchestration

📌 MDM Lead (Uttar Pradesh)
🏢 Emids Technologies
📍 Uttar Pradesh