Responsibilities:
Design and implement complex data pipelines using Dataflow (Apache Beam) and Dataproc (Spark).
Lead Cloud Composer (Airflow) orchestration design, optimizing DAG structure, dependencies, and retry strategies.
Drive advanced Python and SQL implementations, including reusable frameworks and utility components.
Build and standardize custom Dataflow / Beam templates and reusable pipeline patterns.
Perform deep performance tuning for BigQuery (query optimization, partitioning strategies, cost optimization).
Troubleshoot complex pipeline failures and performance bottlenecks across processing and orchestration layers.
Review code and design artifacts produced by engineers; provide technical guidance and mentoring.
Own technical quality for assigned modules, including non-functional aspects (performance, reliability, scalability).




Act as the first point of escalation for technical issues before involving architects.
Collaborate with architects to understand design intent and ensure correct implementation.

Mandatory Technical Skills:
Advanced GCP Knowledge
Solid understanding of IAM strategy, service isolation, and project-level design considerations.
Programming & Querying
Advanced proficiency in Python and complex GoogleSQL.
Orchestration
Robust hands-on expertise with Cloud Composer / Airflow, including best practices and optimization.
Data Processing
Deep hands-on experience with Dataflow and Apache Beam.
Solid working knowledge of Dataproc / Spark internals.
Analytics Platform
Advanced experience with BigQuery, including performance and cost optimization techniques.

📌 Gcp Data Engineer Hyderabad
🏢 Elabs Infotech
📍 Hyderabad