About The Role

We are seeking an experienced Sr Data Engineer with 12-16 years of experience in designing, building, and modernizing enterprise data platforms. The ideal candidate will possess deep expertise in Microsoft Fabric, large-scale data migrations, modern analytics architectures, and real-time data processing. This role will lead the design and implementation of end-to-end data solutions that support analytics, reporting, AI/ML workloads, and strategic business initiatives.

Key Responsibilities

Microsoft Fabric Platform Leadership

- Design, implement, and govern enterprise-scale data platforms using Microsoft Fabric.
- Architect and manage OneLake, Lakehouse, and Data Warehouse workloads.
- Establish workspace governance, security standards, capacity planning, and cost optimization strategies.
- Drive Fabric platform adoption across business and technology teams.
- Define and implement architecture standards, best practices, and operational frameworks.

Data Engineering &

- Migration

- Lead large-scale data migration initiatives from legacy applications and data platforms into Microsoft Fabric.
- Design and develop Data Factory Pipelines and Dataflows Gen2 for batch and incremental data ingestion.
- Create scalable PySpark notebooks and SQL-based transformations to support enterprise workloads.
- Implement source-to-target mapping, data validation, reconciliation, and cutover strategies.
- Develop CDC (Change Data Capture) frameworks for near real-time data synchronization.
- Manage data archival and retention strategies during application decommissioning programs.

Real-Time Data &

- Streaming Solutions

- Design and implement streaming architectures using Fabric Real-Time Intelligence.
- Develop Eventstream integrations and KQL-based solutions for real-time insights.
- Integrate enterprise systems using Kafka and Azure Event Hubs.
- Support migration from legacy middleware and messaging platforms to modern event-driven architectures.
- Build monitoring and alerting capabilities for streaming data pipelines.





Analytics &

- Business Intelligence

- Design contemporary Medallion Architectures (Bronze, Silver, Gold) for enterprise analytics.
- Develop and optimize Power BI semantic models leveraging Direct Lake architecture.
- Create scalable, governed datasets supporting enterprise reporting and AI consumption.
- Improve dataset performance, refresh efficiency, and cost effectiveness.
- Partner with business stakeholders to deliver actionable analytics solutions.

Data Governance &

- Compliance

- Implement Microsoft Purview for enterprise-wide data governance.
- Establish data lineage, cataloging, metadata management, and data quality frameworks.
- Define governance controls to ensure compliance, security, and auditability.
- Collaborate with security, compliance, and enterprise architecture teams on data governance initiatives.

Experience

Required Qualifications

- 12-16 years of experience in Data Engineering, Data Warehousing, BI, or Analytics platform development.
- 5+ years of experience designing cloud-based analytics platforms.
- Proven experience leading large-scale data migration and modernization programs.
- Experience working in enterprise environments supporting mission-critical applications.
- DP-600/DP-700 certification preferred

Technical Skills

- Microsoft Fabric Platform
- OneLake
- Lakehouse Architecture
- Fabric Data Warehouse
- Data Factory Pipelines
- Dataflows Gen2
- Power BI
- Direct Lake
- Spark / PySpark
- Advanced SQL
- Kafka
- Azure Event Hubs
- Change Data Capture (CDC)
- Data Modeling
- Data Reconciliation &
- Validation
- Medallion Architecture
- Microsoft Purview
- Data Lineage &
- Governance

We are an equal opportunity employer committed to fair and ethical hiring practices. We do not charge any fees or accept any form of payment from candidates at any stage of the recruitment process. If anyone claims to offer employment opportunities in our company in exchange for money or any other benefit, please treat it as fraudulent and report it immediately Onselake, Microsoft Fabric Platform,kafka,Lakehouse Architecture,Spark / PySpark

📌 SR LEAD DATA CONSULTANT-INDIA (Bengaluru)
🏢 Lumen Technologies India
📍 Bengaluru