15 Nov
|
IRTISH Consulting Pte
|
Mumbai
15 Nov
IRTISH Consulting Pte
Mumbai
: Confluent Kafka L3 Engineer / Kafka Subject Matter Expert (SME)
Experience Required: 6–10 years overall IT experience, with at least 4–5 years of hands-on experience specifically on Apache Kafka / Confluent Platform in large-scale, mission-critical production environments.
Location / Work Mode: Mumbai, India (Onsite)
Domain: Banking / Financial Services – Payments Applications
About the Role
We are looking for an experienced Confluent Kafka L3 Engineer to own the design, deployment, and production support of a mission-critical, geographically stretched Kafka cluster supporting core banking payment applications.
The candidate will be responsible for ensuring high availability, resiliency, and performance of a Kafka ecosystem servicing ~4.5 crore (45 million) customer accounts with a sustained throughput of 5,000–10,000 TPS.
This is a senior, hands-on role requiring deep platform expertise combined with strong understanding of banking payment workflows and the surrounding technology ecosystem.
Key Responsibilities1. Cluster Architecture & Solutioning
- Design, architect, and optimize Confluent Kafka clusters stretched across four geographically distributed data centers.
- Define topic architecture, partitioning strategy, replication factor, and rack-awareness configurations.
- Provide solutioning inputs, capacity planning, and sizing for current and projected TPS loads.
- Collaborate with enterprise architects and application teams on event-driven and streaming solutions.
1. Deployment & Configuration
- Lead end-to-end deployment of Confluent Kafka components (Brokers, Zookeeper/Kraft, Schema Registry, Kafka Connect, ksqlDB, REST Proxy, Control Center, and MirrorMaker/Cluster Linking).
- Manage cluster upgrades, patching, and version migrations with minimal downtime.
- Implement security configurations (SSL/TLS, SASL/Kerberos, ACLs, RBAC).
- Configure and manage multi-DC replication strategies.
1.
Production Support & Troubleshooting (L3)
- Act as the highest level of technical escalation (L3) for production incidents, providing RCA and permanent fixes.
- Monitor and manage a environment supporting ~4.5 crore accounts with a throughput of 5,000–10,000 TPS.
- Troubleshoot broker failures, ISR issues, consumer lag, partition rebalancing, and replication issues.
- Perform deep-dive performance tuning (JVM, OS, network, producer/consumer).
- Drive proactive capacity planning.
1. Monitoring & Operations
- Set up and manage monitoring, alerting, and observability using Confluent Control Center, Prometheus, Grafana, and Splunk/ELK.
- Track operational metrics like broker health, under-replicated partitions, consumer lag, latency, and throughput.
- Ensure adherence to banking regulatory, audit, and compliance requirements.
- Participate in DR drills, BCP, and periodic failover testing.
1. Ecosystem & Integration Knowledge
- Demonstrate strong understanding of the surrounding banking ecosystem (core banking systems, payment gateways/switches, NPCI/RTGS/NEFT/IMPS/UPI rails, fraud engines, reconciliation systems).
- Partner with cross-functional teams to resolve issues impacting Kafka pipelines.
- Support integration via Kafka Connect, custom producers/consumers, and stream processing frameworks.
1. Documentation & Governance
- Create and maintain runbooks, SOPs, architecture diagrams, and knowledge base articles.
- Enforce platform best practices and governance.
- Mentor L1/L2 support engineers.
Required Skills & QualificationsTechnical Skills
- 8–10 years total IT experience; minimum 4–5 years of dedicated Kafka/Confluent Platform experience.
- Hands-on expertise with Confluent Platform components.
- Proven experience managing stretched/multi-DC Kafka clusters and cross-DC replication.
- Strong understanding of Kafka internals, performance tuning, capacity planning, and troubleshooting.
- Working knowledge of Linux administration, networking fundamentals, and scripting (Shell/Python).
- Familiarity with security implementations (TLS/SSL, SASL, Kerberos, ACLs, RBAC) and monitoring tools.
- Exposure to stream processing frameworks is a plus.
Domain Skills
- Prior experience in banking/payments/fintech environments with an understanding of core banking, payment processing, and settlement/reconciliation workflows.
- Familiarity with regulatory and compliance requirements (RBI guidelines, data security standards, audit trails).
- Understanding of the broader banking technology ecosystem.
Soft Skills
- Strong analytical and problem-solving skills under pressure (P1/P2 situations).
- Excellent communication skills.
- Ability to lead RCA discussions and drive permanent fixes.
- Willingness to participate in on-call rotations.
Preferred / Valuable to Have
- Confluent Certified Administrator / Developer for Apache Kafka certification.
- Experience with cloud-hosted Kafka (Confluent Cloud, MSK, or Azure Event Hubs for Kafka).
- Experience with containerization/orchestration (Docker, Kubernetes) for Kafka deployments.
- Exposure to CI/CD pipelines for Kafka configuration and topic management (GitOps for Kafka).
- Prior experience in a bank, NBFC, or payments company handling similar scale.
Education
- Bachelor's or Master's degree in Computer Science, Information Technology, or related field.
Pay: Up to ₹2,100,000.00 per year
Work Location: In person
📌 Kafka Admin (Mumbai)
🏢 IRTISH Consulting Pte
📍 Mumbai