29 Oct
|
IndiTech Consulting
|
India
29 Oct
IndiTech Consulting
India
Experience: 5+ Years
Work Mode: Remote
Employment Type: Part-Time | Freelance
About the Role
We are looking for an experienced GCP Data Engineer to design, develop, and optimize scalable data pipelines using Python, SQL, and Google Cloud Platform (GCP). The ideal candidate should have strong hands-on experience with BigQuery, Dataflow, Cloud Storage, and Cloud Composer, along with expertise in batch and real-time data processing.
This is an excellent opportunity for data engineering professionals seeking a flexible, part-time remote engagement.
Key Responsibilities
- Design, develop, and maintain ETL/ELT data pipelines using Python, SQL, and GCP services.
- Build and optimize data processing workflows using BigQuery, Dataflow, and Cloud Composer.
- Write complex SQL queries for data transformation, analysis, and performance optimization.
- Implement data ingestion and storage solutions using Google Cloud Storage (GCS).
- Develop batch and real-time data processing pipelines using Pub/Sub and Dataflow.
- Automate and orchestrate workflows using Apache Airflow and Cloud Composer.
- Implement data quality checks,
validation, security, and pipeline monitoring.
- Troubleshoot pipeline failures and optimize performance, scalability, and reliability.
- Collaborate with technical teams to deliver efficient and reliable data solutions.
Required Skills
- Strong hands-on experience with Google Cloud Platform (GCP) data engineering services.
- Proficiency in Python and SQL.
- Experience with Google BigQuery, including query optimization and data transformation.
- Hands-on experience with Google Cloud Storage (GCS).
- Experience developing data pipelines using Dataflow and Cloud Composer.
- Knowledge of Pub/Sub for event-driven and real-time data processing.
- Experience with Apache Airflow for workflow orchestration.
- Understanding of ETL/ELT processes, data modeling, and data quality practices.
Positive to Have
- Experience with PySpark.
- Knowledge of CI/CD pipelines and Git.
- Experience with automated testing, monitoring, and deployment of data pipelines.
📌 Freelance GCP Data Engineer (India)
🏢 IndiTech Consulting
📍 India