Lead / Senior Microsoft Fabric Data Engineer

Location : Bangalore (On-site, with flexibility for 12 remote days)

Job Type : Permanent

Experience Level : 8 years in data engineering overall, with 2 years in a leadership/architect capacity

About the Role :

We are seeking an experienced Lead Microsoft Fabric Data Engineer to drive our initiative to modernize our legacy BI platform onto Microsoft Fabric. This is a hands-on technical leadership role : you will architect the core Fabric solution yourself, set practical engineering standards, and guide a small team through a phased migration. We expect most candidates to bring solid, deep Azure Synapse/ADF/Databricks experience plus 1.52 years of practical, hands-on Microsoft Fabric work.

Key Responsibilities :

1.

Technical

Leadership &

- Hands-On Architecture :
- Own the end-to-end technical design of the Fabric Lakehouse/Warehouse solution, and remain hands-on in build for the first 2-3 quarters.
- Set pragmatic engineering standards and coding conventions for the Fabric platform (naming, folder structure, notebook/pipeline patterns).
- Act as the team's go-to person for Fabric - evaluate new features/capacity options and recommend what's actually useful for our workloads.
- Mentor 2-4 mid-level data engineers day to day on Fabric and Spark/SQL best practices.

2. Lakehouse &

- Warehouse Design :
- Design the Lakehouse/Data Warehouse architecture in Fabric, including medallion (Bronze/Silver/Gold) layout.
- Define workspace and domain structure, full enterprise data-mesh patterns are a plus, not a prerequisite.
- Right-size Fabric capacity (F-SKU) and monitor/optimize compute and storage cost.
- Choose sensibly between Direct Lake, Import, and Direct Query per use case.

3.

Legacy Platform Migration :

- Assess our current BI/DW stack and produce a practical, phased migration plan.
- Lead pilot migrations of 1-2 priority workloads, validate the approach, then scale to the rest.
- Prior experience migrating any legacy platform (SQL Server, Synapse, Teradata, Oracle, Cognos, etc.) to a modern cloud/Fabric target is valued.
- Define simple, trackable success metrics for migration progress and data quality.

4.

Core Data Engineering :

- Build and review data pipelines using Fabric Data Factory, Dataflows Gen2, and Notebooks (PySpark/Spark SQL).
- Implement incremental loads, CDC, and SCD patterns.
- Where relevant, build real-time ingestion with Event streams and KQL Database - deep Real-Time Intelligence expertise is a plus, not a must-have.
- Apply core performance practices : partition design, file/table optimization, query tuning.
- Stand up basic automated data-quality checks (schema,



null/volume checks).

5. Security &

- Governance :
- Implement workspace security, item-level permissions, and row-level security (RLS) for the workloads we own.
- Apply sensible data classification.
- Follow existing company security/compliance policy; flag gaps to the security/compliance team.

6. DevOps &

- Automation :
- Set up Git integration and a working branching strategy for Fabric development.
- Build CI/CD for Fabric deployments using Azure DevOps or GitHub Actions and the Fabric REST APIs.
- Script routine deployment/admin tasks in PowerShell or Python.
- Full infrastructure-as-code (Terraform/Bicep) and Kubernetes are nice-to-have.

7.

Stakeholder Collaboration :

- Work directly with business stakeholders and Power BI developers/analysts to translate requirements into working pipelines and models.
- Report progress and blockers clearly to engineering leadership.
- Run lightweight design reviews with the team rather than formal architecture review boards.

8.

Continuous Improvement :

- Keep the team current on new Fabric features as they roll out.
- Share learnings internally - a short write-up or demo after each major milestone is enough; formal thought-leadership/publishing is a bonus.

Required Skills &

Experience :

Microsoft Fabric (Must-Have) :

- 1.5-2 years of genuine hands-on Fabric experience (production or serious pilot work) - note this is close to the realistic ceiling given Fabric's Nov 2023 GA date.
- Working knowledge of Lakehouse, Warehouse, Data Factory (Fabric), Dataflows Gen2, and Notebooks.
- Practical understanding of OneLake, workspace management, and Fabric capacity basics.
- Comfortable choosing between Direct Lake, Import, and DirectQuery for a given scenario.

Azure &

- Data Platform :
- 8 years in data engineering, including 4 years hands-on with Azure data services (Synapse Analytics, ADF, ADLS, or Databricks).
- Strong PySpark, Spark SQL, Python, and T-SQL skills.
- Solid grasp of Delta Lake and Parquet; distributed-computing fundamentals.
- Most candidates will be Azure Synapse/ADF veterans who moved into Fabric over the last 1-2 years - that profile is exactly what we want.

Architecture &

- Design :




- Experience designing data warehouse/lakehouse solutions for a mid-to-large organization.
- Solid dimensional modelling skills; data vault or data mesh exposure is a plus.
- Basic FinOps awareness for cloud data cost control.

Migration Experience :

- Has led or been a key contributor on at least 1 legacy-to-cloud data platform migration (any source : on-prem SQL Server, Oracle, Teradata, Synapse, Cognos, Tableau, etc.).
- Comfortable doing lightweight current-state assessment and migration planning.

Security &

- Governance :
- Practical experience with RLS/OLS/CLS and workspace-level security in Power BI/Fabric or Synapse.
- General familiarity with data governance concepts (classification, lineage) - deep Purview administration experience is a plus.

DevOps &

- Automation :
- Experience with Azure DevOps or GitHub Actions for CI/CD.
- Working knowledge of Fabric REST APIs, PowerShell, or Python for automation.

Certifications :

- Microsoft Certified : Fabric Analytics Engineer Associate (DP-600).
- Microsoft Certified : Fabric Data Engineer Associate (DP-700).
- Microsoft Certified : Azure Solutions Architect Expert (AZ-305).

Preferred Qualifications :

- Experience in manufacturing, retail, or healthcare data environments.
- Power BI administration and Premium/Fabric capacity management experience.
- Exposure to AI/ML integration with Fabric (Azure ML, Copilot, OpenAI) a strong plus.
- Comfort working in agile/Scrum delivery.

Leadership &

- Soft Skills :
- Able to lead and mentor a small team (2-4 engineers) by example, staying hands-on rather than purely directive.
- Clear communicator who can explain technical trade-offs to both engineers and business stakeholders.
- Comfortable operating with a fair amount of ambiguity - this is a build-as-we-go modernization, not a fully scoped enterprise program.
- Pragmatic, delivery-focused - biased toward shipping a working pilot over designing the perfect long-term architecture upfront.

Key Deliverables :

- Current-state assessment and a phased migration plan for priority workloads.
- Core Lakehouse/Warehouse architecture in Fabric, agreed with the team.
- Workload migrated and validated end-to-end.
- Baseline engineering standards, Git/CI-CD setup, and initial security model in place.

Why Join Us? :

This is a chance to build a modern Fabric-based data platform largely from scratch, with real ownership over the architecture and hands-on influence from day one. You'll work on one of Microsoft's newest and fastest-evolving data platforms, grow into a broader platform-leadership role as the team scales, and help shape how the organization does data engineering going forward.

📌 Lead/Senior Microsoft Fabric Data Engineer (India)
🏢 Harpoon Technology u0026 Consultancy
📍 India