10 Sep
|
Zorba consulting
|
Hyderabad
10 Sep
Zorba consulting
Hyderabad
- We are looking for an experienced GCP Data Engineer with 5+ years of experience in data engineering and hands-on expertise in Google Cloud Platform.
- The candidate will be responsible for implementing, managing, monitoring, and troubleshooting data ingestion pipelines using GCP services.
- The role requires robust experience with GCS, BigQuery, Dataflow, Cloud Composer, Data Fusion, and Dataproc, along with proficiency in Python, Terraform, and Git.
- The ideal candidate should be comfortable working independently on data ingestion, ETL/ELT, migration, and production support activities.
Key Responsibilities
- Configure and execute data ingestion pipelines using reusable GCP-based frameworks.
- Develop and manage ETL/ELT workflows using GCS, BigQuery, Dataflow, Composer, Data Fusion, and Dataproc.
- Manage ingestion parameters, job configurations, metadata, and runtime dependencies.
- Monitor daily data pipeline executions and ensure agreed SLAs are met.
- Troubleshoot and resolve data, parameter, configuration, and job execution issues.
- Analyze runtime failures and independently identify and implement appropriate solutions.
- Escalate framework-level defects to the core development team when required.
- Support data engineering and migration projects on GCP.
- Develop scripts and automation using Python.
- Maintain infrastructure configurations and deployment scripts using Terraform.
- Follow Git-based version control and deployment best practices.
- Collaborate with cross-functional teams to ensure smooth delivery and production operations.
- Identify opportunities to improve ingestion frameworks, processes,
and operational efficiency.
- Contribute to framework enhancements and automation initiatives where applicable.
- Work with TWS/TWSd or other enterprise scheduling tools for job scheduling and monitoring, where required.
Mandatory Skills
- 5+ years of experience as a Data Engineer or in a similar role.
- Strong hands-on experience with Google Cloud Platform (GCP).
- Hands-on experience with: Google Cloud Storage (GCS) BigQuery Dataflow Cloud Composer Cloud Data Fusion Dataproc Strong Python skills for scripting, debugging, and automation.
- Experience with Terraform / Infrastructure as Code (IaC).
- Good understanding of Git and version control.
- Strong understanding of data ingestion, ETL/ELT workflows, and file formats.
- Ability to independently troubleshoot and resolve production/runtime data issues.
- Strong analytical and problem-solving skills.
Good to Have
- Experience with TWS/TWSd or other enterprise job scheduling and monitoring tools.
- Understanding of BigQuery SQL optimization.
- Experience in GCP data migration projects.
- Exposure to reusable data ingestion frameworks and production data operations.
- Preferred Candidate Profile The ideal candidate should be a hands-on GCP Data Engineer who can independently manage data ingestion pipelines, troubleshoot production issues, work with multiple GCP data services, and support ETL/ELT operations.
- Strong Python, Terraform, and Git knowledge is expected, along with the ability to work effectively in a production environment.
Location: Pan India-Bengaluru,Hyderabad,Delhi / NCR,Chennai,Pune,Kolkata,Ahmedabad,Mumbai
📌 GCP Data Engineer (Hyderabad)
🏢 Zorba consulting
📍 Hyderabad