Location: PAN India
Experience: 4+ Years
Work Mode: Remote
Job Description
We are looking for an experienced GCP Data Engineer with 4+ years of hands-on experience in Google Cloud Platform to work on an ongoing cloud data integration project.
The candidate will work closely with an existing SnapLogic integration project, so candidates with prior SnapLogic experience will be preferred. However, SnapLogic is not mandatory for candidates with robust ETL, data integration, API, and cloud data engineering experience and the ability to quickly learn new integration technologies.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines on GCP.
- Work with BigQuery, Google Cloud Storage (GCS), Dataflow, Pub/Sub, Cloud Composer/Airflow and other GCP services.
- Develop data ingestion and transformation workflows from multiple source systems.
- Perform data transformation, validation, cleansing, and reconciliation using SQL and Python.
- Develop and maintain Airflow DAGs for workflow orchestration and scheduling.
- Work with batch and near real-time data processing pipelines.
- Troubleshoot pipeline failures and production data issues.
- Collaborate with integration and data engineering teams working on the existing SnapLogic environment.
- Understand existing integration workflows and quickly adapt to SnapLogic where required.
- Participate in code reviews, testing, deployment, and production support activities.
Required Skills
- 4+ years of experience in Data Engineering / GCP
- Strong hands-on experience with Google Cloud Platform
- Good experience with:
- BigQuery
- GCS
- Dataflow
- Pub/Sub
- Cloud Composer / Apache Airflow
- Python
- SQL
- Strong understanding of ETL/ELT and data integration concepts
- Experience working with multiple source systems and data formats such as JSON, CSV, XML, etc.
- Good problem-solving and troubleshooting skills.
- Ability to understand and work with new integration technologies