Manage data engineering projects, ensuring alignment with business objectives. Provide strategic guidance on data engineering best practices. Oversee a team of data engineers. Ensure continuous improvement of data processes.
- Writing code in programming language & working experience in Python, PySpakrk, Scala or Similar
- Big Data Framework & working experience in Spark or Hadoop or Hive (incl. derivatives like pySpark (prefered), SparkScala or SparkSQL) or Similar
- Data warehouse working experience of concepts and development using SQL on single (SQL Server, Oracle or Similar) and parallel platforms (Azure SQL Data Warehouse or Snowflake)
- Code Management & working experience in GIT Hub, Azure DevOps or Similar
Manage data engineering projects. Provide strategic guidance. Oversee data engineering team. Ensure continuous improvement of data processes.
Basic understanding of :
- Scheduling and workflow management & working experience in either ADF, Informatica, Airflow or Similar
- Enterprise Data Modelling and Semantic Modelling & working experience in ERwin, ER/Studio, PowerDesigner or Similar
- Logical/Physical model on Big Data sets or up-to-date data warehouse & working experience in ERwin, ER/Studio, PowerDesigner or Similar
- Agile Process (Scrum cadences, Roles, deliverables) & basic understanding in either Azure DevOps, JIRA or Similar
- Architecture and data modelling for Data Lake on cloud & working experience in Amazon WebServices (AWS), Microsoft Azure, Google Cloud Platform (GCP)
- Basic understanding of Build and Release management & working experience in Azure DevOps, AWS CodeCommitt or Similar
Graduate in Computer Science, Data Science, or related field. 3-4 years of experience in data engineering or a related field.
Candidate have at least 4-6 years hands-on Azure experience as a Data Engineer, must be an expert in SQL and have extensive expertise building data pipelines. Candidate will be accountable for meeting deliverable commitments including schedule and quality compliance. This Candidate must have skills to plan and schedule own work activities, coordinate activities with other cross-functional team members to meet project goals.