8 to 12 years of experience
Skills: Python, Spark, SQL, PostgreSQL , CI/CD practices
Role Overview
NAB is building a bespoke data platform focused on Financial Crime transaction analysis. The platform processes tens of millions of transaction records daily and enables large-scale analytics to identify suspicious patterns and behavioral anomalies.
We are looking for a hands-on Data Engineer with solid Python and Spark expertise who can design scalable data pipelines, perform large-scale data analysis, and contribute to the architecture of a high-volume data platform.
This role is ideal for engineers who enjoy deep coding, distributed data processing, and building custom data platforms rather than relying purely on managed tools.
Key Responsibilities
Build and maintain scalable data pipelines across stages such as data download, ingestion, and analysis within a bespoke data platform.
Develop high-performance data processing logic using Python and PySpark.
Perform large-scale transaction data analysis using custom algorithms and in-house logic to detect financial crime patterns.
Work with high-volume datasets (80–90 million records processed regularly) and optimize pipeline performance.
Design efficient data models and transformations for large-scale processing.