Big Data Engineer – GCP & Scala

Location: Bengaluru and Pune

Experience: 5–10 Years

Apply here: https://forms.cloud.microsoft/r/Vngu7esJju

[email protected]

We are looking for an experienced Data Engineer with strong expertise in Google Cloud Platform (GCP) , Scala , and Apache Spark to design, develop, and maintain scalable data engineering solutions. The ideal candidate will have hands-on experience building cloud-native data pipelines, processing large-scale datasets, and implementing modern data architectures using GCP services and big data technologies.

Key Responsibilities

- Design, develop, and optimize scalable batch and real-time data pipelines using Scala and Apache Spark .
- Build and maintain ETL/ELT workflows for large-scale data processing.
- Develop cloud-native data solutions on Google Cloud Platform (GCP) .
- Work with Hadoop ecosystem technologies, including Hadoop , Hive , and HDFS .
- Develop and optimize SQL queries and database solutions using PostgreSQL and other relational databases.
- Implement workflow orchestration using Apache Airflow/Cloud Composer .




- Build CI/CD pipelines using Jenkins and follow DevOps best practices.
- Apply cloud design patterns and architectural best practices to deliver secure, scalable, and high-performing solutions.
- Collaborate with cross-functional teams including Data Architects, Business Analysts, Product Owners, and DevOps engineers.
- Monitor, troubleshoot, and optimize production data pipelines and cloud resources.
- Ensure data quality, governance, security, and performance standards are met.

Required Skills
- Strong experience with Scala programming.
- Hands-on expertise in Apache Spark .
- Experience with Google Cloud Platform (GCP) .
- Strong knowledge of Hadoop , Hive , and HDFS .
- Experience with Apache Airflow .
- Hands-on experience with Jenkins .
- Solid SQL development and query optimization skills.
- Experience with PostgreSQL .
- Understanding of cloud architecture and cloud design patterns.
- Experience designing and implementing scalable ETL/data pipeline solutions.

📌 Big Data Developer (Pune)
🏢 CAPCO
📍 Pune