Role Overview

We are looking for a highly skilled Kafka Migration Engineer to join the ETO Factory team and support a high-visibility on-premises Apache Kafka to AWS MSK migration project .

The candidate will be responsible for Kafka cluster discovery, workload assessment, capacity planning, AWS MSK configuration, migration execution, testing, validation, performance tuning, automation, and legacy Kafka cluster decommissioning .

The ideal candidate should have strong hands-on experience with Apache Kafka, AWS MSK, distributed messaging systems, cloud migration, Linux, programming, automation, and CI/CD , along with the ability to collaborate with application, infrastructure, and platform teams.

Key Responsibilities

- Assess on-premises Kafka clusters and perform discovery across:
- Topics
- Partitions
- Consumer groups
- Throughput patterns
- Retention policies
- Inter-service dependencies
- Evaluate workloads based on:
- Message volume
- Business criticality
- Ordering requirements
- Latency requirements
- Consumer group complexity
- Determine appropriate AWS MSK cluster placement, sizing, and configuration .
- Plan and provision AWS MSK infrastructure, including:
- Broker sizing
- Storage
- Networking
- Security
- Execute UAT and production parallel testing using MirrorMaker 2 or similar replication tools.
- Validate message delivery outcomes and troubleshoot migration discrepancies.




- Perform Kafka/MSK performance tuning and optimization to ensure scalability and production readiness.
- Collaborate with application, infrastructure, and platform teams to finalize:
- Migration plans
- Producer/consumer cutover strategies
- Validation criteria
- Rollback plans
- Execute production cutover activities, including:
- Consumer group migration
- Offset synchronization
- Post-release validation
- Support successful decommissioning of legacy on-premises Kafka clusters .
- Design and implement automation frameworks for repeatable, large-scale Kafka-to-MSK migrations.
- Develop scripts and utilities for:
- Topic creation
- ACL migration
- Consumer group management
- Migration validation
- Explore AI-assisted engineering and automation approaches to improve migration efficiency and quality.
- Document migration procedures, risks, operational learnings, and best practices.

Experience
- 3–5 years of hands-on engineering experience in distributed messaging systems.
- Strong experience working in collaborative, team-based engineering environments.

Programming Strong proficiency in at least one of:
- Python
- Java
- Shell scripting
- Similar programming/scripting languages

Systems & DevOps
- Robust Unix/Linux fundamentals.
- Experience troubleshooting application, deployment, environment, and runtime issues.
- Understanding of:
- SDLC
- CI/CD
- Change Management
- Kubernetes-based deployments

📌 Kafka Migration Engineer (India)
🏢 Deloitte
📍 India