We are looking for a skilled and motivated Data Engineer to join our team supporting Commonwealth Bank of Australia (CBA) . The ideal candidate will have hands-on experience in designing, developing, and optimizing scalable data pipelines using contemporary big data technologies and cloud platforms. You will work closely with cross-functional teams to build reliable data solutions that support business intelligence, analytics, and regulatory reporting.

Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines.
- Develop and optimize Spark and Scala applications for large-scale data processing.
- Build and manage data solutions using Hadoop technologies including HDFS and Hive.
- Design, implement, and optimize Snowflake data warehouse solutions.
- Write high-performance SQL queries and perform query optimization.
- Develop cloud-native data pipelines using AWS services such as S3, Glue, Athena, and Redshift.
- Perform data validation, quality checks, and troubleshooting of production issues.
- Collaborate with business analysts, architects,



and development teams to understand data requirements.
- Manage source code using GitHub and automate deployments through GitHub Actions and TeamCity.
- Create and maintain Shell scripts for automation and operational support.
- Participate in release management, production support, and continuous improvement initiatives.
- Follow banking security, governance, and compliance standards.

Required Skills
- Strong experience with Spark and Scala
- Hands-on experience with Hadoop (HDFS, Hive)
- Experience working with Snowflake
- Advanced SQL skills including performance tuning and query optimization
- Experience with AWS services:
- Amazon S3
- AWS Glue
- Amazon Athena
- Amazon Redshift
- Experience in ETL/Data Pipeline development
- Version control using GitHub
- CI/CD using GitHub Actions and TeamCity
- Shell Scripting
- Experience working in the Banking domain

Preferred Skills
- Experience in Financial Crim

📌 Data Engineer (India)
🏢 D2kss Software Services
📍 India