Description

We are looking for an experienced Senior Data Engineer – Microsoft Fabric with 8–10 years of overall data engineering experience and strong hands-on expertise in building, optimizing, and administering modern cloud data platforms.

The candidate should have strong experience across Microsoft Fabric, PySpark, Spark SQL/T-SQL, Delta Lake, OneLake and Lakehouse architecture, with the ability to combine hands-on engineering with strategic platform administration.

The role will be responsible for designing scalable data solutions, developing enterprise-grade ETL/ELT pipelines, optimizing Fabric workloads, and supporting platform governance, security, capacity management, monitoring, and DevOps practices. These responsibilities reflect the internal Fabric requirement, including ownership of architecture alongside governance and platform administration.

Key Responsibilities

1. Microsoft Fabric Data Engineering

- Design and develop scalable data engineering solutions using Microsoft Fabric Lakehouse, Warehouse, OneLake, Notebooks, Pipelines, and Dataflows Gen2.
- Design and implement Medallion Architecture using bronze, silver, and gold layers for enterprise data workloads.
- Develop reusable ETL/ELT frameworks for ingestion, transformation, validation, and consumption.
- Integrate structured and semi-structured data from enterprise databases, APIs, cloud storage, SaaS applications, and other data sources.
- Build scalable batch-processing solutions and data integration patterns using Fabric components.

2. PySpark, SQL & Delta Lake

- Develop scalable transformations using PySpark, Spark SQL, Python and SQL for enterprise-scale datasets. Similar capabilities are reflected in existing internal Fabric/Data Engineer profiles.
- Design and maintain Delta Lake/Delta Table solutions supporting reliable and scalable data processing.
- Implement incremental processing, CDC/SCD patterns, data validation, reconciliation, and data-quality frameworks.
- Optimize Spark workloads, SQL queries, Delta tables, notebooks, and data pipelines for performance and resource utilization.
- Apply appropriate partitioning, caching and query optimization techniques to large-scale workloads.

3. Strategic Microsoft Fabric Platform Administration

- Support administration and governance of the Microsoft Fabric tenant and workspaces.




- Configure and periodically review tenant settings and platform configurations aligned with organizational security and governance requirements
- Monitor Fabric Capacity/F-SKU consumption using Fabric capacity monitoring capabilities and help optimize workloads to balance cost and performance.
- Identify capacity-intensive or competing workloads and recommend workload optimization, scaling, or scheduling strategies.
- Establish platform-level standards, engineering best practices, workspace conventions, deployment controls, and operational processes.
- Drive standardization and platform adoption across engineering and analytics teams.

4. Security & Governance

- Implement and maintain appropriate access-control models including RBAC, RLS and OLS within the Fabric ecosystem.
- Support integration with Microsoft Purview for metadata, discovery, lineage, classification, and enterprise data governance
- Support governed handling of sensitive enterprise datasets.
- Work with security and architecture teams to establish appropriate data-access and governance standards.
- Participate in architecture, security, governance, and operational reviews.

5. Performance, Monitoring & Cost Optimization

- Proactively monitor platform utilization and data workloads.
- Troubleshoot performance issues in Fabric pipelines, Spark notebooks, SQL workloads, Lakehouses, Warehouses, and semantic workloads.
- Build operational monitoring and alerting approaches for critical data workloads.
- Identify opportunities to reduce unnecessary compute/storage utilization and improve overall platform efficiency.
- Support proactive platform reliability and production incident/root-cause analysis.

6. CI/CD & DevOps

- Implement Git-based source control and CI/CD/deployment practices for Microsoft Fabric workloads.
- Establish controlled promotion of artifacts across Development, Test and Production environments.
- Automate deployments where appropriate and maintain appropriate versioning and release-management disciplines.
- Perform peer reviews and enforce engineering standards for notebooks, SQL, pipelines, and related artifacts.

7.



Technical Leadership

- Provide technical guidance to Data Engineers and developers on Microsoft Fabric engineering standards.
- Conduct design reviews, code reviews, performance reviews, and technical troubleshooting.
- Collaborate with Data Architects, BI teams, security teams, application teams and business stakeholders to translate requirements into scalable data solutions. Similar cross-functional responsibilities are part of the existing Fabric role definition.
- Mentor junior engineers and promote reusable engineering patterns and platform best practices.
- Contribute to technical roadmaps and platform modernization initiatives

Enable Skills-Based Hiring No ECMS REQ IDAdditional Details

- ECMS REQ ID : 582479
- Project Location 1 : TELANGANA | HYDERABAD
- Project Location 2 : (No Value)
- Project Location 3 : (No Value)
- Project Location 4 : (No Value)
- Delivery SPOC : Manish_deshpande01
- Relevant Experience : 8-10 Years
- Mandatory skills : • 8–10 years of Data Engineering / Data Platform experience• Strong hands-on experience with Microsoft Fabric• Strong hands-on development using PySpark• Advanced SQL / T-SQL / Spark SQL• Strong Delta Lake knowledge• OneLake and Fabric Lakehouse• Microsoft Fabric Data Factory Pipelines• Dataflows Gen2• Fabric Notebooks• ETL/ELT architecture and development• Medallion Architecture• Data modeling and Data Warehousing concepts• Git and CI/CD• Data governance and security principles• Performance tuning and troubleshooting• Platform monitoring and administration
- Desired skills : Positive to Have• Power BI and Semantic Models• Direct Lake• Microsoft Purview• Azure Data Factory• Azure Databricks• ADLS Gen2• Azure DevOps / GitHub• Azure Monitor / Log Analytics• Real-Time Analytics / Eventstream• Data quality and metadata-management frameworksPreferred Certifications• DP-600: Microsoft Fabric Analytics Engineer Associate• AZ-305: Azure Solutions Architect Expert
- Domain (Industry) : Insurance
- BGC (Before onboarding / After onboarding) : HybridOB
- Total Experience (Ex. 5-7 Years) : 8 to 10 years
- BGC Details : Infosys Standard
- BGC Vendor : Infosys Standard
- Mode of Interview : Face to Face
- WFO / WFH / Hybrid : Hybrid
- Please enter shift timings : 2:00 PM to 11:00 PM IST
- Shift Timings : Afternoon

📌 582479_DNAINS_Senior Data Engineer – Microsoft Fabric_India (Hyderabad)
🏢 Trigent Software
📍 Hyderabad