Job Title: Developer
Work Location: Chennai / Bangalore /Hyderabad
Skill Required: Digital : BigData and Hadoop Ecosystems~Digital : Kafka~Digital : Google Cloud
Experience Range: 10+ Years
Job Description:
Big Data with GCP Tech Lead:
1. 10+ Years of experience in developing and migrating Big Data applications in to GCP
2. Should be very strong in Big Data skill set Pyspark , Hive , Hadoop and HQL
3. Hands-on experience with core GCP services relevant to data engineering, including Dataproc, BigQuery, Cloud Storage, Dataflow, Pub/Sub, and IAM.
4. Proven expertise in developing, optimizing, and deploying complex PySpark applications for large-scale data processing.
5. Create scalable data pipelines and ETL/ELT processes to extract, transform, and load data from various sources into GCP services.
6. Build and optimize data models, queries, and storage solutions using GCP tools like BigQuery for efficient data analysis and reporting.
7. Utilize tools like Cloud Composer and Apache Airflow to automate data workflows and processes
8. Analyze, refactor,
and optimize existing PySpark code for efficient execution within GCP environments, potentially leveraging services like Dataproc, Dataflow, or BigQuery Spark.
9. Design, develop, and deploy scalable and robust data pipelines on GCP using services such as Cloud Storage, BigQuery, Pub/Sub, Dataflow, and Dataproc.
10. Oversee and execute the migration of large-scale datasets from source systems to GCP storage solutions (e.g., Cloud Storage, BigQuery).
11. Create detailed documentation for migrated systems and establish best practices for PySpark development and data engineering on GCP.
12. Excellent analytical and problem-solving skills to diagnose and resolve complex technical issues.
13. Should have experience in working in Agile Methodology
14. Should be flexible in timings
GCP Engineers
1. 7+ Years of experience in developing and migrating Big Data applications in to GCP
2. Should be very robust in
📌 GCP Data Engineer (India)
🏢 Clifyx
📍 India