">GPC Data Engineer

3-5 Years Bengaluru

- GCP
- Python
- Spark
- SQL

Data Engineer:

- Should have total 3-5 Yrs of experience in Data Engineering.
- Should have hands-on experience on Distributed computing
- Should have working experience on Data Architecture design
- Strong on programming languages like Python, Java.
- Should be aware of storage and compute options and when to choose what
- Must have one cloud hands-on experience
- Proficient in either Apache Spark or Apache Beam or Apache Flink
- Advance SQL knowledge is mandatory
- Must be aware of Streaming concepts like Windowing , Late arrival , Triggers etc
- Should have exposure on GCP tools to develop end to end data pipeline for various scenarios (including ingesting data from traditional data bases as well as integration of API based data sources).
- Should have Business mindset to understand data and how it will be used for BI and Analytics purposes.
- Should have working experience on CI/CD pipelines, Deployment methodologies, Infrastructure as a code




- Should have advance understanding of different file formats e.g. AVRO, Parquet, ORC , CSV and when to choose what
- Positive to have, Hands-on experience on Kubernetes
- Good to have Vector based Database like Qdrant
- Should have good understanding on Cluster Optimisation/ Pipeline Optimisation strategies

Experience in Working with GCP tools like

Storage : CloudSQL , Cloud Storage, Cloud Bigtable, Bigquery, Cloud Spanner, Cloud DataStore, Vector database

Ingest : Pub/Sub, Cloud Functions, AppEngine, Kubernetes Engine, Kafka, Micro services

Schedule : Cloud Composer, Airflow

Processing: Cloud Dataproc, Cloud Dataflow, Apache Spark, Apache Flink

CI/CD : Bitbucket+Jenkinjs / Gitlab

Infrastructre as a tool : Terraform

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

📌 GPC Data Engineer (Karnataka)
🏢 Quarks Technosoft
📍 Karnataka