Job Description Data Engineer (Scala Spark + GCP)

Job Title

Data Engineer Scala Spark GCP

Location

Bangalore (Manyata Tech Park)

Work Model

Hybrid (3 Days Work From Office)

Experience

4 to 6 Years

Interview Mode

Face-to-Face (F2F)

FG IDs

- IBMFG2JP00041762
- IBMFG2JP00041761

Job Summary

We are seeking a highly skilled Data Engineer with strong expertise in Scala, Apache Spark, and Google Cloud Platform (GCP) to design, develop, optimize, and maintain large-scale data processing pipelines. The ideal candidate should have hands-on experience in Dataproc, BigQuery, Cloud Storage, and performance optimization of distributed data processing applications.

Key Responsibilities

- Design and develop scalable data pipelines using Scala and Apache Spark.
- Build, deploy, and optimize Spark applications on GCP Dataproc.
- Develop and maintain data ingestion, transformation, and processing frameworks.
- Create and optimize complex SQL queries in BigQuery.
- Implement ETL/ELT workflows for large-scale data processing.
- Work with Google Cloud Storage (GCS) for data storage and management.
- Develop automation scripts using Unix/Linux Shell Scripting.
- Participate in code reviews and follow development best practices.
- Manage source code using Git and support CI/CD implementation.
- Troubleshoot performance issues and optimize Spark jobs.
- Collaborate with cross-functional teams to understand business and technical requirements.
- Maintain technical documentation and operational procedures.

Mandatory Skills

Data Engineering

- Data Engineering Experience: 4-6 Years
- ETL/ELT Development




- Data Pipeline Design & Optimization

Scala & Spark

- Scala Programming
- Apache Spark Core
- Spark SQL
- Spark Performance Tuning
- Distributed Data Processing
- Spark Optimization Techniques

Google Cloud Platform (GCP)

- Google Cloud Platform (GCP)
- Dataproc
- BigQuery
- Google Cloud Storage (GCS)

Database & SQL

- Advanced SQL
- BigQuery SQL
- Query Optimization
- Data Modeling

Operating Systems

- Linux
- Unix
- Shell Scripting

DevOps & Version Control

- Git
- CI/CD Pipeline
- Jenkins/GitLab CI (Preferred)

Nice-to-Have Skills

- Python Development
- Data Governance
- Data Quality Frameworks
- Metadata Management
- Cloud Migration Projects
- Agile/Scrum Methodology

Required Candidate Profile

4-6 Years of Strong Data Engineering Experience

Hands-on experience with Scala-based Spark Development

Robust GCP experience including Dataproc and BigQuery

Excellent SQL Query Writing and Performance Tuning Skills

Experience with Unix/Linux Scripting

Good Understanding of Git and CI/CD Processes

Strong Problem-Solving & Debugging Skills

Excellent Communication Skills

Available for Face-to-Face Interviews in Bangalore

Willing to work in Hybrid Model (3 Days WFO at Manyata)

Keywords for Job Posting

Scala, Apache Spark, Spark SQL, GCP, Google Cloud Platform, Dataproc, BigQuery, Google Cloud Storage, GCS, Data Engineer, ETL, ELT, SQL, Linux, Unix, Shell Scripting, Git, CI/CD, Python, Data Governance, Data Quality, Metadata Management, Spark Optimization, Distributed Computing, Cloud Data Engineering.

📌 Data Engineer (Scala Spark + GCP)_ Immediate jiner_5+ Yrs_BanLor_F (Bengaluru)
🏢 DynPro
📍 Bengaluru