24 Sep
|
Algoworks
|
Noida
Role overview
Role: Senior Microsoft Fabric Performance Scalability Specialist Location: Remote, India.
Experience: 10+ Years
We are seeking a hands-on Senior Microsoft Fabric Performance Scalability Specialist with 10+ years of Data Engineering and Data Warehousing experience to optimize, scale, develop, and support an enterprise Microsoft Fabric platform supporting 1,500+ clients/tenants.
The role will focus on Fabric performance optimization, multi-tenant scalability, Fabric development, new data-source onboarding, client onboarding and support, production issue resolution, and SLA management.
The ideal candidate must have strong hands-on experience with Microsoft Fabric, Fabric Data Warehouse, Lakehouse, PySpark, Python, SQL performance tuning, and Medallion Architecture (Bronze/Silver/Gold). This is a hands-on development and performance optimization role focused on production delivery rather than primarily architecture or administration.
Key responsibilities
1. Microsoft Fabric Performance Scalability
- Optimize Microsoft Fabric performance, scalability, concurrency, and capacity utilization across Fabric Data Warehouse, Lakehouse, OneLake, Shortcuts, Pipelines, and Notebooks.
- Design and enhance the platform to efficiently support migration and ongoing processing for 1,500+ tenants/clients.
- Develop scalable processing strategies for high-volume and low-volume tenants while meeting data-load and reporting SLAs.
- Analyze Fabric capacity/CU utilization, throttling, concurrency, Query Insights, and workload performance.
- Implement performance optimization recommendations based on workload analysis.
- Conduct performance, scalability, concurrency, stress, and load testing.
- Establish measurable performance benchmarks and continuously improve platform performance.
2. Fabric Development Data Processing
- Develop and enhance Fabric notebooks using PySpark, Python, and SQL for scalable full and incremental data processing.
- Optimize tenant grouping, batching, parallel processing, concurrency control, workload prioritization, watermark processing, retries, and failure recovery.
- Optimize Fabric Data Warehouse SQL queries, data loads, joins, table structures, and CLUSTER BY strategies.
- Develop reusable and scalable data-processing frameworks for enterprise workloads.
- Support production-grade development and optimization of Fabric-based data solutions.
3. New Data Source Development Onboarding
- Work closely with ETL/Data Engineering teams to onboard new data sources and new sets of tables into Microsoft Fabric.
- Analyze source schemas, tables, relationships, data volumes, dependencies, primary keys, and incremental-load requirements.
- Perform required development, code changes, configuration, and testing in Microsoft Fabric for newly introduced data sources and tables.
- Develop and enhance Fabric notebooks, pipelines, SQL code, metadata/control frameworks, Lakehouse objects, and Fabric Data Warehouse tables.
- Configure and test full and incremental data-loading processes for newly onboarded tables.
- Perform unit, integration, data-validation, and performance testing before production deployment.
- Coordinate with ETL teams to validate source-to-target mappings, transformation logic, dependencies, and data completeness.
- Integrate recent data sources into the established Bronze Silver Gold Lakehouse/Fabric Data Warehouse architecture.
- Develop reusable and metadata-driven patterns to reduce development effort for future data-source and table onboarding.
4. Client Onboarding Production Support
- Work directly with clients and internal teams during onboarding, migration, production cutover, and post-production support.
- Understand client data volumes, processing requirements, dependencies, and SLA expectations.
- Support client onboarding activities including full loads, incremental-load enablement, data validation, performance validation, and production cutover.
- Investigate and resolve client-reported data, performance, data-load, and reporting issues.
- Perform end-to-end root-cause analysis across source systems, Medallion layers, Fabric notebooks, pipelines, Lakehouse, Data Warehouse, and reporting workloads.
- Troubleshoot missing or duplicate data, failed or delayed loads, incremental-processing issues, performance degradation,
and SLA violations.
- Implement permanent fixes and enhance shared frameworks to prevent recurring issues across tenants.
5. Production Operations SLA Management
- Monitor production workloads and ensure data-processing and reporting SLAs are consistently achieved.
- Identify performance bottlenecks, capacity constraints, processing failures, and concurrency issues.
- Prioritize and resolve production incidents based on business impact and SLA requirements.
- Collaborate with ETL, Data Engineering, reporting, and client-facing teams to resolve complex production issues.
- Document root-cause analysis, corrective actions, and preventive measures.
- Continuously improve production stability, reliability, and operational efficiency.
Required technical skills and competencies
- 10+ years of experience in Data Engineering, Data Warehousing, ETL/ELT, or enterprise data platforms.
- Strong recent hands-on Microsoft Fabric production experience.
- Proven experience in Microsoft Fabric performance optimization and scalability.
- Strong hands-on experience with Fabric Data Warehouse, Lakehouse, OneLake, Pipelines, and Notebooks.
- Advanced PySpark, Python, and SQL development and performance-tuning skills.
- Strong production experience with Medallion Architecture (Bronze/Silver/Gold) - mandatory.
- Experience developing and optimizing large-scale full and incremental data-loading frameworks.
- Experience handling high-volume, concurrent, and multi-tenant workloads.
- Strong understanding of Fabric capacity, CU utilization, concurrency, throttling, and workload management.
- Hands-on experience with new data-source and table onboarding, development, testing, and production deployment.
- Experience working collaboratively with ETL/Data Engineering teams to implement source-to-target data pipelines.
- Experience working directly with clients during onboarding, migration, and production issue resolution.
- Strong troubleshooting, root-cause analysis, communication, and problem-solving skills.
Disclaimer: This job posting and location has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.
📌 Sr Microsoft Fabric Performance & Scalability Specialist (Noida)
🏢 Algoworks
📍 Noida