10 Aug
|
Zorba Consulting India
|
Hyderabad
10 Aug
Zorba Consulting India
Hyderabad
We are looking for an experienced Kafka Data Engineer with strong hands-on expertise in Apache Kafka, Kafka Topic/Cluster Migration, Data Streaming, and Distributed Systems. The candidate will be responsible for planning and executing Kafka migrations, cluster-to-cluster replication, data validation, performance tuning, and ensuring minimal downtime during migration activities.
Key Responsibilities
- Design and execute Kafka topic and cluster migrations across environments, clusters, or cloud platforms.
- Perform Kafka cluster upgrades, broker migrations, and data replication using MirrorMaker 2.0, Confluent Replicator, Cluster Linking, or similar technologies.
- Migrate and validate topics, schemas, consumer groups, offsets, ACLs, and configurations.
- Monitor Kafka throughput, latency, consumer lag, partition health, and overall cluster performance.
- Troubleshoot Kafka connectivity, replication, partition balancing, and consumer lag issues.
- Develop automation scripts using Python, Shell scripting, Java, or Scala.
- Work with application teams to validate producers, consumers,
and streaming pipelines after migration.
- Implement Kafka security using SSL/TLS, SASL, Kerberos, RBAC, and ACLs.
- Collaborate with DevOps, Cloud, and Data Engineering teams for infrastructure and deployment activities.
- Prepare migration plans, rollback strategies, technical documentation, and post-migration reports.
Mandatory Skills
- Strong hands-on experience with Apache Kafka Administration and Development.
- Proven experience in Kafka Topic Migration / Cluster Migration / Cross-Cluster Replication.
- Hands-on experience with MirrorMaker 2.0, Confluent Replicator, or Cluster Linking.
- Strong knowledge of Kafka Connect, Kafka Streams, Schema Registry, and Confluent Platform.
- Good understanding of Kafka partitions, brokers, replication, consumer groups, offsets, and consumer lag.
- Experience working in Linux/Unix environments.
- Proficiency in Python, Java, or Scala.
- Robust u
📌 Kafka Data Engineer_5+ years (Hyderabad)
🏢 Zorba Consulting India
📍 Hyderabad