TNB Tenaga Malaysia to migrate its existing Cloudera data ecosystem to the up-to-date Databricks Data Intelligence Platform (Lakehouse Architecture). This migration is not a mere technical uplift, but a critical enabler aligned directly with TNB's Reimagining TNB 2025 (RT25) strategy and its overarching Energy Transition (ET) Aspiration Plan, specifically supporting the Grid of the Future and Sustainability Pathway 2050 pillars.TNB?s existing Cloudera setting (v7.1.7) to the Databricks Lakehouse Platform using a ?Lift-and-Shift, then Modernize? approach.

This approach ensures minimal operational disruption during migration while establishing a scalable foundation for advanced analytics, AI, and domain-led modernization.
Lead Data Engineer
Role Summary

Leads the design, development, migration, and deployment of data pipelines, ETL processes, and data engineering best practices for the Databricks platform.
Key Responsibilities




Lead migration of HiveQL, Impala, Spark SQL, Bash, and Informatica workloads.
Design and build batch and streaming pipelines using Databricks.
Develop PySpark-based transformation frameworks.
Implement Delta Live Tables and Databricks Workflows.
Establish CI/CD standards and coding best practices.
Conduct performance tuning and optimization.
Support production deployments and hypercare activities.
Mentor and guide development teams.
Required Skills
Databricks Platform
Apache Spark / PySpark
Delta Live Tables (DLT)
Databricks Workflows
Structured Streaming
SQL, Python, Scala
Informatica Migration
Git, CI/CD Pipelines
Data Quality and Testing Frameworks
Agile Delivery Methodologies

📌 Lead Data Engineer Pyspark Bengaluru (India)
🏢 Happiest Minds Technologies
📍 India