Senior Technical Specialist

Bengaluru, Karnataka

Job Summary

Hands-on build (the core of the role)

- Design and build data pipelines in Microsoft Fabric — Data Factory pipelines, Copy activities, and Spark notebooks (PySpark) — across a bronze/silver/gold medallion architecture.
- Write production-grade PySpark and SQL for ingestion, transformation, cleansing, and star-schema modelling in Delta Lake / OneLake.
- Build and optimise semantic models (Direct Lake), including relationships, DAX measures, and row-level security, tuned for performance against the gold layer.
- Design and implement metadata-driven ingestion frameworks — config-driven, parameterised pipelines that scale across many sources rather than one pipeline per table.
- Implement robust handling for incremental loading, schema drift/evolution, and data quality, including CDC/watermark/hash-based patterns as appropriate per source.
- Integrate and configure governance via Microsoft Purview — lineage, cataloguing, classification — as a first-class part of the build, not an afterthought.

Technical leadership

- Own the technical architecture and set the design patterns, coding standards, and best practices the team builds against.
- Lead code reviews, establish CI/CD for Fabric (Git integration, deployment pipelines, environment/workspace strategy), and enforce version control discipline.
- Make and document the key build-vs-buy and architectural trade-off decisions (lakehouse vs. warehouse, batch vs. streaming, load strategies).
- Partner with data governance, infrastructure, and security on workspace design, access, networking (e.g. Private Endpoints), and capacity/cost management.

Delivery & mentorship

- Translate business and analytical requirements into a scoped,



sequenced technical roadmap.
- Mentor and grow data engineers — pairing, design guidance, and raising the team's Fabric capability.
- Manage delivery within a sprint/Agile cadence (Azure DevOps), balancing hands-on contribution with unblocking the team.
- Communicate status, risks, and trade-offs clearly to technical and non-technical stakeholders.

Key Responsibilities

- Robust hands-on experience building data platforms on Microsoft Fabric (or very recent Azure Synapse / Databricks + Power BI experience directly transferable to Fabric).
- Expert PySpark and strong SQL ; production experience with Delta Lake and the medallion architecture .
- Proven experience designing and building data pipelines — both orchestration (Data Factory / pipelines) and transformation (Spark notebooks) — and knowing when to use which.
- Hands-on semantic modelling and DAX , ideally with Direct Lake , including performance optimisation.
- Solid grasp of data warehousing and dimensional modelling (star schema, SCD, surrogate keys).
- Experience with incremental load patterns, schema drift handling, and data quality frameworks.
- CI/CD and Git for data platforms; workspace/environment management.
- A track record of technical leadership — setting standards, reviewing code, mentoring — while remaining hands-on.

Skill Requirements

- Python, PySpark, Meddalion Architect, Microsoft Fabric, Data Pipeline, Data Engg.
- Azure ecosystem (ADLS, Key Vault, networking/Private Endpoints, Azure DevOps).
- Real-Time Intelligence / streaming (Eventstream, KQL) exposure.
- Domain experience in [your industry — e.g. financial services / BFSI].
- Relevant certification (e.g. DP-600 / DP-700 Fabric , or Azure Data Engineer).

Other Requirements

📌 Senior Technical Specialist (India)
🏢 HCLTech
📍 India