Role Summary

We are looking for a GCP Data Engineering Manager / AD with 8 to 15+ years of experience to lead the delivery and optimization of cloud-based data platforms. The role combines strong expertise in GCP data services with hands-on performance engineering for large-scale Spark and Dataproc workloads.

Key Responsibilities

GCP Data Engineering and Platform Delivery

- Design and deliver data solutions using Cloud Run, Cloud Run Functions with Python, Cloud SQL PostgreSQL, Pub/Sub, GCS and BigQuery.
- Build and manage orchestration workflows using Cloud Composer / Managed Airflow.
- Drive Airflow and Composer optimization, including FinOps, capacity planning and cost governance.
- Support Databricks-based data processing and integration.
- Design and integrate REST APIs across cloud applications and data platforms.
- Provide technical leadership, architecture guidance and delivery oversight to engineering teams.

Performance Engineering and Dataproc Optimization





- Execute workload test waves and enhance Python-based execution templates.
- Troubleshoot Spark, YARN and Dataproc failures using Spark UI and platform diagnostics.
- Analyze approximately 60 workload profiles and capture cost-performance benchmarks.
- Define N1/N2 mapping strategies and translate custom machine requirements into standard GCP machine types.
- Recommend VM sizing, schema design and performance improvements for Big Data workloads.

Required Skills

- 8 to 15 years in Data Engineering, with solid experience on GCP.
- Python, Spark, Dataproc, GCS, BigQuery and Cloud Data Architecture.
- Cloud Run, Cloud Functions, Cloud SQL PostgreSQL and Pub/Sub.
- Cloud Composer / Airflow administration, optimization and FinOps.
- Spark UI debugging, YARN diagnostics and workload benchmarking.
- Strong stakeholder management, team leadership and solution design capabilities.

📌 Manager | GCP Data Engineer | Ahmedabad | Engineering | Data Modernization & Migration (India)
🏢 Deloitte
📍 India