- We are seeking an experienced Kafka Administrator to manage and maintain enterprise level Apache Kafka clusters
- The ideal candidate will be responsible for installation configuration monitoring performance tuning and ensuring high availability of Kafka infrastructure in distributed environments
- Key Responsibilities
- Kafka Cluster Administration
- Install configure and maintain Apache Kafka clusters
- Manage Kafka brokers topics partitions and replication
- Perform cluster upgrades scaling and capacity planning
- Monitoring Performance Tuning
- Monitor Kafka cluster health using tools like
- Prometheus Grafana Confluent Control Center
- Optimize broker performance throughput and latency
- Tune configurations for producers and consumers
- Data Streaming Integration
- Manage real time data pipelines using Kafka
- Handle integration with
- Spark Hadoop Flink
- Databases APIs
- Support streaming use cases and event driven architecture
- Security Governance
- Implement authentication and authorization SSL SASL Kerberos
- Configure ACLs Access Control Lists
- Ensure secure data transmission and compliance
- Replication High Availability
- Ensure fault tolerance and data replication across brokers
- Configure multi cluster setups MirrorMaker Cluster Linking
- Handle disaster recovery strategies
- Troubleshooting Support
- Diagnose and fix issues like
- Broker failures
- Consumer lag
- Performance bottlenecks
- Analyze logs and perform root cause analysis
- Automation DevOps
- Automate deployments using Ansible Terraform or scripts
- Manage CI CD pipelines for Kafka environments
- Version control using Git
- Zookeeper KRaft Management
- Manage Zookeeper clusters if applicable
- Configure Kafka in KRaft mode Zookeeper less architecture
Technical Requirements:
- Primary skills Kafka administration
Additional Responsibilities:
- Experience with Confluent Platform
- Knowledge of Kubernetes Kafka on K8s
- Experience with DevOps tools and CI CD pipelines
- Certification in Kafka Confluent nice to have