19 Nov
|
PhonePe
|
Bengaluru
Job Description
About the Role
We are building an in-house team to maintain and contribute to our internal Apache Kafka distribution. This is not an operations role — you will work on the Kafka codebase itself: the broker, the KRaft controller, the protocol, the clients, and the storage layer. You will be our technical anchor for all things Kafka internals, driving upstream contributions, backporting critical fixes, and shaping our internal distribution to meet the needs of a large-scale production fleet.
We already do this successfully with GlusterFS, where we maintain an in-house distribution backed by deep upstream expertise. We want to replicate that model for Kafka.
Key Responsibilities
Own our internal Kafka distribution: branch management, patch curation, release cutting, upgrade planning
Contribute upstream to Apache Kafka — bug fixes, KIPs, code reviews, design discussions
Diagnose and fix deep broker/controller issues: replication, partition reassignment,
KRaft consensus, log storage, ISR management, protocol handling
Build custom patches for production needs that aren't suitable for upstream (performance tuning for our workload profile, observability hooks, operational safety valves)
Establish engineering practices for Kafka development within the team: how we test, how we benchmark, how we validate patches before rolling them out
Mentor a supporting engineer and grow the team's capability over time
Act as the internal authority on Kafka internals — advise platform teams on configuration, capacity planning, and architectural decisions that require knowledge of broker behavior
Act as the primary interface for audit and compliance teams, ensuring PhonePe Kafka distribution meets various compliance and regulatory standards
📌 Staff Engineer Kafka Codebase And Internals Bengaluru
🏢 PhonePe
📍 Bengaluru