30 Aug
|
Questhiring
|
Gurugram
30 Aug
Questhiring
Gurugram
Data Engineer II – GCP Data Platform Location: Gurgaon
Work Mode: Hybrid
Experience: 4+ Years
About the Role
We are looking for a hands-on Data Engineer II with strong GCP experience to build scalable data pipelines, transformations, and reusable data-platform capabilities.
The role involves taking solutions end-to-end—from design and development to testing, deployment, and production support —while building reliable and reusable data engineering solutions.
Key Responsibilities
Design, develop, and maintain production-grade data pipelines on GCP .
Build scalable batch and incremental processing solutions.
Develop complex transformations using Big Query, SQL, and Dataform/dbt .
Build and maintain workflows using Cloud Composer / Apache Airflow .
Develop reusable transformation components, frameworks, and orchestration patterns.
Build distributed data-processing solutions using Dataflow / Apache Beam and/or Dataproc / Spark .
Design data models for analytical and downstream data products.
Implement data-quality checks, validation, reconciliation, and data lineage.
Optimize pipelines and Big Query workloads for performance and cloud cost .
Implement automated testing and integrate data workloads with CI/CD .
Monitor production pipelines and troubleshoot issues through root-cause analysis.
Collaborate with Data Engineering, Platform Engineering, Dev Ops, Architecture, and business teams.
Must-Have Skills
~4+ years of hands-on Data Engineering experience.
~ Strong hands-on experience with GCP and Big Query .
~ Advanced SQL – complex joins, CTEs, window functions,
and query optimization.
~ Strong Python development skills.
~ Experience with production ETL/ELT pipelines.
~ Hands-on experience with Cloud Composer / Apache Airflow .
~ Experience with Dataform and/or dbt .
~ Experience with Dataflow / Apache Beam or Dataproc / Spark / PySpark .
~ Strong understanding of data modelling and large-scale datasets.
~ Experience with incremental and idempotent processing.
~ Understanding of data contracts, schema evolution, data quality, and reconciliation.
~ Experience with Git, automated testing, and CI/CD .
~ Solid production troubleshooting and problem-solving skills.
Technical Skills
Cloud: GCP, Big Query, Google Cloud Storage
Programming: Python, SQL
Data Processing: Dataflow, Apache Beam, Dataproc, Spark/PySpark
Orchestration: Cloud Composer, Apache Airflow
Transformation: Dataform, dbt
Engineering: Git, CI/CD, Automated Testing, Monitoring
Data Engineering: ETL/ELT, Batch Processing, Data Modelling, Data Quality, Data Contracts, Metadata & Lineage
Good to Have
Apache Iceberg or modern lakehouse technologies.
Streaming or event-driven processing.
Data Product / Data Mesh concepts.
Metadata-driven or configuration-driven processing.
Change Data Capture (CDC).
Terraform / Infrastructure as Code.
Experience building reusable components for multiple engineering teams.
What We’re Looking For
A strong ownership mindset, good analytical and debugging skills, ability to independently design solutions, and the confidence to take data engineering solutions through production.
📌 Data Engineer (Machine Learning) (Gurugram)
🏢 Questhiring
📍 Gurugram