05 Nov
|
CodeVyasa
|
Mumbai
We are looking for experienced Confluent Kafka Developer | Mumbai having 8+yrs of experience.
About Us
CodeVyasa is a mid-sized product engineering company that works with top-tier product and solutions organizations such as McKinsey, Walmart, RazorPay, Swiggy, and others. We are a team of 550+ engineers, driving innovation across Product & Data Engineering, focusing on Agentic AI, RPA, Full Stack, and GenAI-based solutions.
Key Responsibilities
- Cluster Architecture & Solutioning: Design, architect, and optimize Confluent Kafka clusters stretched across four geographically distributed data centers. Define topic architecture, partitioning strategy, replication factor, and rack-awareness configurations. Provide solutioning inputs, capacity planning, and sizing for current and projected TPS loads. Collaborate with enterprise architects and application teams on event-driven and streaming solutions.
- Deployment & Configuration: Lead end-to-end deployment of Confluent Kafka components (Brokers, Zookeeper/Kraft, Schema Registry, Kafka Connect, ksqlDB, REST Proxy, Control Center, and MirrorMaker/Cluster Linking). Manage cluster upgrades, patching, and version migrations with minimal downtime. Implement security configurations (SSL/TLS, SASL/Kerberos, ACLs, RBAC). Configure and manage multi-DC replication strategies.
- Production Support & Troubleshooting (L3): Act as the highest level of technical escalation (L3) for production incidents, providing RCA and permanent fixes. Monitor and manage a environment supporting ~4.5 crore accounts with a throughput of 5,000–10,000 TPS. Troubleshoot broker failures, ISR issues, consumer lag, partition rebalancing,
and replication issues. Perform deep-dive performance tuning (JVM, OS, network, producer/consumer). Drive proactive capacity planning.
- Monitoring & Operations: Set up and manage monitoring, alerting, and observability using Confluent Control Center, Prometheus, Grafana, and Splunk/ELK. Track operational metrics like broker health, under-replicated partitions, consumer lag, latency, and throughput. Ensure adherence to banking regulatory, audit, and compliance requirements. Participate in DR drills, BCP, and periodic failover testing.
- Ecosystem & Integration Knowledge: Demonstrate strong understanding of the surrounding banking ecosystem (core banking systems, payment gateways/switches, NPCI/RTGS/NEFT/IMPS/UPI rails, fraud engines, reconciliation systems). Partner with cross-functional teams to resolve issues impacting Kafka pipelines. Support integration via Kafka Connect, custom producers/consumers, and stream processing frameworks.
- Documentation & Governance: Create and maintain runbooks, SOPs, architecture diagrams, and knowledge base articles. Enforce platform best practices and governance. Mentor L1/L2 support engineers.
Required Skills & Qualifications
- Technical Skills:
8–10 years total IT experience; minimum 4–5 years of dedicated Kafka/Confluent Platform experience. Hands-on expertise with Confluent Platform components. Proven experience managing stretched/multi-DC Kafka clusters and cross-DC replication. Strong understanding of Kafka internals, performance tuning, capacity planning, and troubleshooting. Working knowledge of Linux administration, networking fundamentals, and scripting (Shell/Python). Familiarity with security implementations (TLS/SSL, SASL, Kerberos, ACLs, RBAC) and monitoring tools. Exposure to stream processing frameworks is a plus.
- Domain Skills: Prior experience in banking/payments/fintech environments with an understanding of core banking, payment processing, and settlement/reconciliation workflows. Familiarity with regulatory and compliance requirements (RBI guidelines, data security standards, audit trails). Understanding of the broader banking technology ecosystem.
- Soft Skills: Strong analytical and problem-solving skills under pressure (P1/P2 situations). Excellent communication skills. Ability to lead RCA discussions and drive permanent fixes. Willingness to participate in on-call rotations.
Why Join CodeVyasa?
- Work on innovative, high-impact projects with leading global clients.
- Exposure to modern technologies, scalable systems, and cloud-native architectures.
- Continuous learning and upskilling opportunities through internal and external programs.
- Supportive and collaborative work culture with flexible policies.
- Competitive salary and comprehensive advantages package.
- Free healthcare coverage for employees and dependents.
📌 Confluent Kafka Developer (Mumbai)
🏢 CodeVyasa
📍 Mumbai