Data Contracts; Data Transformation; Data Validation; Data Observability Module; Ingestion Standardization; Master Data Management (data side); Data Toolkit Design; Standardised data platform architecture; Common CLI/SDK; Synthetic Data Generator
Location
Prudential VCC Bangalore (preferred)
Role summary
Complete remaining engineering and LBU rollout of the UDP data toolkit and standardised ingestion, working on Azure Databricks and GCP BigQuery.
Key Responsibilities
Key responsibilities
Build and roll out ingestion standardisation patterns and connectors (file, API, database) using procured ingestion tools.
Implement and roll out the Data Validation, Data Transformation, Data Contracts, Platform/Data Observability modules and the common CLI/SDK, including versioning and support.
Integrate the Synthetic Data Generator on UDP and support PACS deployment.
Support MDM integration with UDP and downstream applications.
Write Python/SQL and Spark/BigQuery code with unit and integration tests; use Git workflows and CI/CD.
Execute discovery, prerequisites, deployment,
acceptance and hypercare for each LBU rollout using playbooks.
Document runbooks, known issues and release records in Prudential-approved repositories.
Skill Requirements
Must-have skills and experience
3–5 years of data engineering experience with Azure Databricks (Spark, Delta) and/or GCP BigQuery.
Robust Python and SQL; experience with dbt and Airflow (or Data Factory / Composer).
CI/CD with GitHub/Azure DevOps and basic Terraform.
Understanding of data quality, schema evolution and data contract concepts.
Good to have
Experience with Kafka, Fivetran, Informatica or Reltio integrations.
Experience building reusable SDKs or CLIs (Python packaging, semantic versioning).
Exposure to data observability tooling.
Certifications (within 90 days of assignment)
Databricks Data Engineer Associate/Skilled
GCP Skilled Data Engineer or Azure DP-203 equivalent
HashiCorp Terraform Associate (preferred)
Must Have Skill
GCP BigQuery
Good to Have Skill
Other Requirements
📌 Senior Technical Lead Bengaluru
🏢 HCLTech
📍 Bengaluru
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.