As a GCP data engineer the colleague should be able to designs scalable data architectures on Google Cloud Platform, using services like Big Query and Dataflow. They write and maintain code (Python, Java), ensuring productive data models and seamless ETL processes. Quality checks and governance are implemented to maintain accurate and reliable data.
n
Security is a priority, enforcing measures for storage, transmission, and processing, while ensuring compliance with data protection standards. Collaboration with cross-functional teams is key for understanding diverse data requirements. Comprehensive documentation is maintained for data processes, pipelines, and architectures.
n
Responsibilities extend to optimizing data pipelines and queries for performance, troubleshooting issues, and proactively monitoring data accuracy. Continuous learning is emphasized to stay updated on GCP features and industry best practices, ensuring a current and effective data engineering approach.
n
Key Skills:
n
- Proficiency in programming languages: Python, Java
- Expertise in data processing frameworks: Apache Beam (Data Flow)
- Hands-on experience with GCP services: Big Query, Dataflow
- Knowledge of data modeling and database design
- Experience in ETL (Extract, Transform, Load) processes
- Familiarity with cloud storage solutions
- Strong problem-solving abilities in data engineering challenges
- Understanding of data security and scalability
- Proficiency in relevant tools like Apache Airflow