We are looking for an experienced Senior Data Engineer with strong expertise in Microsoft Fabric, Databricks, PySpark, and modern data engineering technologies. The ideal candidate will be responsible for designing scalable data pipelines, migrating legacy SQL Server workloads, and building robust lakehouse solutions using Medallion Architecture.
Key Responsibilities :
- Design and develop Bronze, Silver, and Gold data pipelines using Microsoft Fabric and Databricks.
- Implement Cascade 1.0 & 2.0 merge logic in the Silver layer, including handling new and null columns.
- Migrate legacy SQL Server tables, views, stored procedures, and SSIS ETL workloads into Microsoft Fabric notebooks and pipelines.
- Develop PySpark transformations and Delta Lake models with a focus on performance and cost optimization.
- Build data quality checks, data lineage, and source-to-target reconciliation processes.
- Collaborate with BI developers, report engineers,
and QA teams to deliver governed semantic models.
- Design and implement scalable and reliable data engineering solutions following best practices.
- Optimize data pipelines and processing workloads for performance, scalability, and cost efficiency.
Required Skills :
- Microsoft Fabric - Lakehouse, Pipelines, OneLake
- Azure Databricks
- PySpark
- SQL / T-SQL
- Delta Lake
- Azure Data Factory (ADF)
- Power BI - Datasets / Semantic Models
- Medallion Architecture - Bronze, Silver, Gold
- Data Pipeline Development
- SQL Server and ETL Migration
- Robust understanding of data engineering concepts and cloud-based data platforms.
Preferred Experience :
- Experience migrating legacy SQL Server and SSIS workloads to Microsoft Fabric.
- Hands-on experience with Microsoft Fabric notebooks and pipelines.
- Experience with data quality, reconciliation, and lineage.
- Experience working with enterprise BI and reporting teams.
📌 Senior Data Engineer - Microsoft Fabric/Databricks (India)
🏢 Vy systems
📍 India
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.