Staff Engineer Kafka Codebase And Internals Bengaluru

Staff Engineer Kafka Codebase And Internals Bengaluru

19 Nov
|
PhonePe
|
Bengaluru

19 Nov

PhonePe

Bengaluru

Job Description

About the Role

We are building an in-house team to maintain and contribute to our internal Apache Kafka distribution. This is not an operations role — you will work on the Kafka codebase itself: the broker, the KRaft controller, the protocol, the clients, and the storage layer. You will be our technical anchor for all things Kafka internals, driving upstream contributions, backporting critical fixes, and shaping our internal distribution to meet the needs of a large-scale production fleet.

We already do this successfully with GlusterFS, where we maintain an in-house distribution backed by deep upstream expertise. We want to replicate that model for Kafka.

Key Responsibilities
Own our internal Kafka distribution: branch management, patch curation, release cutting, upgrade planning
Contribute upstream to Apache Kafka — bug fixes, KIPs, code reviews, design discussions
Diagnose and fix deep broker/controller issues: replication, partition reassignment,



KRaft consensus, log storage, ISR management, protocol handling
Build custom patches for production needs that aren't suitable for upstream (performance tuning for our workload profile, observability hooks, operational safety valves)
Establish engineering practices for Kafka development within the team: how we test, how we benchmark, how we validate patches before rolling them out
Mentor a supporting engineer and grow the team's capability over time
Act as the internal authority on Kafka internals — advise platform teams on configuration, capacity planning, and architectural decisions that require knowledge of broker behavior
Act as the primary interface for audit and compliance teams, ensuring PhonePe Kafka distribution meets various compliance and regulatory standards

📌 Staff Engineer Kafka Codebase And Internals Bengaluru
🏢 PhonePe
📍 Bengaluru

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: staff engineer kafka codebase and internals bengaluru / bengaluru

Subscribe to this job alert:

Get the latest job offers by email for: staff engineer kafka codebase and internals bengaluru / bengaluru