Senior Data Engineer — Kafka & GCP (Event-Driven Platform) | US Hours (Shift)
Location: India · Workplace type: 100% in-Office

Build a mission-critical event platform from the ground up — and own it.
We're replacing our Legacy enterprise system with a up-to-date event-driven architecture on Google Cloud, and we're hiring the engineer who will design and build it: a managed Apache Kafka backbone processing ~10 million events per day, real-time rule processing, and a compliant 7-year event archive on BigQuery and Cloud Storage.
This is a rare greenfield: no legacy cloud code to inherit, real scale from day one, and decisions that are yours to make and defend. You'll partner directly with a senior legacy functional expert who owns everything on the Legacy side. You bring deep Kafka, GCP, and Python engineering; we bring the business and legacy domain knowledge and a clear mandate to ship.
What you'll build
A Google Cloud Managed Service for Apache Kafka cluster: topics, partitioning strategy, security, retention




A Python event-ingestion service on Cloud Run receiving events from Legacy and external logistics carriers
A Kafka consumer rule engine maintaining live state in Firestore and triggering actions back into external system
Kafka Connect pipelines archiving every event to BigQuery and to a locked, audit-grade Cloud Storage vault
The monitoring, dead-letter handling, and reconciliation that make "zero events lost" a provable fact, not a hope
What you bring
5+ years in backend or data platform engineering
2+ years running Apache Kafka in production (any distribution — open source, Confluent, MSK, Aiven): you can explain partitions, keys, consumer groups, offsets, and delivery semantics from lived experience
2+ years hands-on GCP: Cloud Run or GKE, IAM and service accounts, VPC fundamentals
Strong production Python: services, not just scripts
BigQuery fluency, including partitioned and clustered table design
A reliability mindset: idempotency, retries, dead-letter queue

📌 GCP Kafka Integration Engineer (Kalyan)
🏢 Olympas
📍 Kalyan