14 Nov
|
Geektrust
|
Bengaluru
14 Nov
Geektrust
Bengaluru
What you need:
7–12 years of relevant experience in Data Engineering, with robust hands-on expertise in Apache Spark and Scala.
Robust experience with GCP services, particularly Dataproc, BigQuery and Google Cloud Storage (GCS).
Solid SQL skills, particularly BigQuery SQL, for data transformation, validation and reconciliation.
Hands-on experience with large-scale ETL pipelines, distributed data processing, Hive, Kafka and high-volume batch/streaming workloads.
Strong Unix/Linux shell scripting skills for automation, data movement and pipeline orchestration, along with familiarity with Git, Jenkins and CI/CD.
Strong understanding of data modelling, data quality, distributed systems and performance optimisation, with the ability to independently troubleshoot complex issues and deliver solutions with minimal handholding.
What you would do:
Design, develop and optimise large-scale data pipelines using Apache Spark and Scala across high-volume batch and streaming workloads.
Build and support data processing solutions using Hive, Kafka, BigQuery, Dataproc and GCS across multiple data sources.
Develop, maintain and optimise ETL workflows, data orchestration pipelines and engineering automation using shell scripting and relevant tools.
Troubleshoot complex data, performance and production issues; perform Spark job tuning and optimise processing across large datasets.
Drive cloud migration, data migration and platform modernisation initiatives, including workload migration, data reconciliation and validation.
Take end-to-end ownership of assigned deliverables, from understanding existing systems and solution development through testing, code reviews, CI/CD deployments and production support, collaborating with cross-functional teams.
📌 Senior Data Engineer Spark, Scala 9 To 13 Yoe Us Based Large Retail Organisation Bengaluru Immediate/early Joiners
🏢 Geektrust
📍 Bengaluru