Introduction

The founding of Metropolitan dates to as early as 1898 and Momentum traces its roots to being established in 1966. Momentum Group was formed from the merger of Metropolitan and Momentum, two sizeable and diverse insurance-based financial services companies in South Africa and was listed on the Johannesburg Stock Exchange on 1 December 2010.

In order to grow its international footprint and diversify its revenue streams, Momentum Group actively pursues financially viable opportunities across the world. Aditya Birla Health Insurance Company Limited, a joint venture with Aditya Birla Capital Limited, was incorporated in 2015 and was our first expansion into India.

Having experienced India first-hand as a hub of both talent and opportunity, we set up Momentum Services in 2017, a wholly-owned Mumbai-based global in-house centre to leverage IT, ITES and Business support services to all our businesses worldwide.

Role Purpose

The incumbent is responsible for designing, developing and optimising data engineering
solutions on the Azure Databricks platform using PySpark, implementing medallion
architecture patterns (Bronze, Silver, Gold), scheduling and orchestrating data workflows,
and delivering quality analytics and reporting solutions to the business

Duties & Responsibilities

• Participate in the analysis, design, development, troubleshooting and support of the
enterprise Databricks data platform and analytics environment
• Design, build, test and implement data pipelines using PySpark and Databricks
notebooks, following medallion architecture patterns (Bronze, Silver, Gold) to
transform raw data into curated, analytics-ready datasets
• Develop and maintain data solutions using PySpark, SQL, Databricks Workflows, Unity
Catalog and Delta Lake on the Azure Databricks platform
• Integrate with diverse source systems (including but not limited to: In-House, Vendorbased,



On-prem and Cloud-based, and Office 365)
• Configure, schedule and monitor Databricks Workflows (jobs) to ensure reliable and
timely data processing, and collaborate with DevOps on CI/CD practices for notebook
and pipeline deployments
• Responsible for the day-to-day data engineering tasks including developing and
optimising PySpark transformations, managing Delta tables, and maintaining data
quality across the medallion layers
• Maintain and evolve the medallion architecture data models, Unity Catalog
governance structures, and analytics-ready Gold layer datasets
• Apply Spark performance tuning techniques (partitioning, caching, broadcast joins,
cluster sizing) to optimise data pipeline throughput and cost efficiency
• Engage directly with business stakeholders to gather requirements, provide datadriven recommendations, and translate business needs into technical solutions. Assist
lead developer in Coordinate team efforts to achieve business objectives (Strategic
and operational)
• Ensure business continuity documentation through Azure DevOps
• Enforce data security and governance standards through Unity Catalog, access
controls and data classification, ensuring adherence by all team members
• Review code implementations, PySpark notebooks and pipeline designs by the Data
team, ensuring adherence to best practices and coding standards
• Oversee quality of data engineering solutions by junior team members, providing
constructive feedback and guidance




• Actively mentor Junior and Intermediate team members, sharing knowledge on
Databricks, PySpark, medallion architecture and data engineering best practice

Drive implementation of cutting-edge data platform capabilities and Databricks features
(e.g., Delta Live Tables, Databricks SQL, ML integrations) through the Lead Developer
and Enterprise Architecture team
• Utilize junior members in achieving large scale project developments
and implementations in consultation with the lead developer

Requirements

• Exposure to the full data engineering and analytics development life cycle.
• 5+ yrs of core experience in Data Engineering, Data Analytics, or Business Intelligence.
• Azure Databricks and PySpark experience essential; experience with Databricks
Workflows (job scheduling and orchestration) required
• Strong knowledge of PySpark, Python, SQL, and data pipeline orchestration using
Databricks Workflows
• Good understanding of medallion architecture (Lakehouse), Delta Lake, data
modelling, and supporting areas (Data Transformation, Governance and Reporting)
• Power BI or Databricks SQL for reporting and analytics preferred

Competencies

Mandatory Skills:
• Exposure to the full data engineering and analytics development
life cycle
• Experience managing Data Projects
• Azure Databricks and PySpark experience essential; experience
with Databricks Workflows (job scheduling and orchestration)
required
• The successful candidate should have experience with medallion
architecture (Bronze, Silver, Gold), Delta Lake, and Unity Catalog;
SQL Server experience advantageous
Nice to have exp/skills:
• Power BI or Databricks SQL for reporting and analytics preferred
• Understanding insurance / accounting principles will be highly
beneficial

📌 Sr Data Engineer - Guardrisk (Navi Mumbai)
🏢 Momentum Group
📍 Navi Mumbai