Description

Description As a GCP Data Engineer, you will be instrumental in modernizing our data infrastructure by building robust, automated ETL/ELT pipelines and leveraging Google Cloud Platform's powerful suite of big data tools. Collaborating closely with Enterprise Architects, you will spearhead the migration from legacy Hadoop/CDP systems to BigQuery. Your daily focus will involve architecting historical and incremental data loads, orchestrating tasks with Apache Airflow, and creating scalable, data-driven solutions that empower the organization with clean, accessible, and organized data.

Responsibilities

Responsibilities

- Pipeline Development: Develop and maintain efficient ETL/ELT pipelines, implementing and automating tasks using Apache Airflow.
- Data Architecture: Architect historical and incremental data loads, continuously evaluating and refining the data architecture to ensure optimal performance.
- Data Warehousing: Support, organize, and optimize data within Google BigQuery data warehouses.
- System Migration: Work alongside Enterprise Architects to lead and deliver the migration of data from legacy Hadoop/Cloudera Data Platform (CDP) systems to BigQuery.
- Enterprise Solutions: Design and build end-to-end, GCP data-driven solutions for enterprise data warehouses and data lakes.

Qualifications

Must Have:

- Certification:



Professional GCP Data Engineer Certification or equivalent.
- Development Experience: 2+ years of coding experience in Java/Python and Infrastructure as Code using Terraform.
- GCP Expertise: 2+ years of experience working in GCP-based Big Data deployments (both Batch and Real-Time), specifically leveraging BigQuery, Bigtable, Google Cloud Storage, Pub/Sub, Data Fusion, Dataflow, Dataproc, and Airflow.
- Data Design: Proven history of designing and delivering comprehensive data lake and data warehousing solutions.
- Data Processing: Strong hands-on experience in extracting, loading, transforming (ETL/ELT), cleaning, and validating data, as well as designing pipelines and architectures for data processing.
- Database Skills: Proficiency in at least one SQL language.
- Big Data Tools: Practical experience working with Spark services.
- Methodologies: Experience working within Agile and Lean methodologies.

Valuable to Have (Preferred):

- Data Visualization: Experience using visualization tools such as Qlik or Looker Studio.
- Migration Experience: Proven experience in migrating legacy systems (preferably Big Data Cloudera Data Platform) into GCP technologies.
- Large-Scale Systems: Experience working with either a MapReduce or an MPP (Massively Parallel Processing) system at any size or scale.

📌 Software Engineer (India)
🏢 Recognized
📍 India