Job Title: GCP Lead Data Engineer Total Experience: 8–13 years Work Location: Bangalore Notice Period: Immediate to 30 Days Joiners Only Role Overview We are seeking a hands-on Lead Data Engineer with strong technical expertise in Google Cloud Platform (GCP), Scala, PySpark, and Kafka.

In this role, you will design, build, and optimize enterprise-grade batch and streaming data processing frameworks.

You will lead technical implementations to enable advanced analytics, machine learning, and AI solutions for global clients while adhering to cloud security, performance, and governance best practices.

Key Responsibilities Pipeline Engineering: Build, maintain, and optimize robust batch and real-time streaming data pipelines using Scala, PySpark, Apache Spark, and Kafka/Pub

Sub.

Architecture & Leadership: Drive technical architecture discussions, code reviews, and Dev

Ops standards across the data engineering team.

Cross-Functional Collaboration: Partner with Data Scientists, Business Analysts, and stakeholders to deliver data platforms that support analytics and machine learning workflows.

Operations & Quality: Ensure high data quality, pipeline reliability, and optimal performance across dynamic distributed systems.

Production Support & Optimization: Perform root cause analysis on production issues, optimize resource costs,



and implement strict cloud security and governance standards.

Required Core Skills Mandatory Technical Stack: Scala, PySpark, and Apache Kafka integrated with core Data Engineering workflows.

Data Engineering Expertise: Proven track record in designing high-throughput ETL/ELT pipelines and processing large-scale datasets.

Distributed Systems: Deep understanding of distributed systems engineering, data modeling, performance tuning, and Apache Beam.

Cloud & Warehousing: Hands-on experience with SQL, NoSQL systems, and cloud data platform ecosystems (Big

Query, GCP).

Software Development Practices: Solid experience with CI/CD automation, version control (Git), and Agile delivery methodologies.

Acceptable Alternative Skill Profiles Candidates matching any of the following technical skill combinations are also eligible: GCP Scala (with or without Kafka) GCP Apache Spark Scala Apache Spark Kafka GCP (Big

Query) PySpark SQL (Kafka is a plus) Preferred (Good-to-Have) Skills Exposure to building real-time streaming architectures using Google Cloud Pub/Sub alongside Kafka.

Python proficiency combined with Scala background.

Familiarity with machine learning data pipelines, feature stores, and MLOps integrations.

📌 GCP Lead Data Engineer (India)
🏢 Halcer
📍 India