Job Summary

We are looking for a Kafka Engineer with 2 to 5 years of experience to design, develop, and optimize high-throughput, low-latency data platforms. The ideal candidate will have hands-on expertise in Apache Kafka, Redis, and MongoDB, with a strong understanding of real-time event streaming, distributed caching, NoSQL databases, and performance optimization. You will work closely with application developers, DevOps engineers, and cross-functional teams to build scalable and resilient data solutions.

Key Responsibilities

Apache Kafka (Event Streaming & Messaging)

- Design, implement, and maintain high-throughput Kafka topics, partitions, and consumer groups.
- Configure Kafka producers and consumers for optimal performance and reliability.
- Ensure message delivery guarantees and manage schema evolution using Confluent Schema Registry.
- Monitor and troubleshoot Kafka clusters, consumer lag, and broker performance.

Redis (Distributed Caching & In-Memory Data Store)

- Design and implement effective caching strategies to improve application performance and reduce database load.
- Utilize Redis data structures such as Hashes, Lists, Sets, Sorted Sets, HyperLogLogs, and Geospatial indexes.
- Configure and manage Redis Clusters, Sentinel deployments, Pub/Sub messaging, and persistence mechanisms.
- Optimize cache performance and resolve issues such as cache stampedes and memory fragmentation.

MongoDB (NoSQL Database)

- Design, implement, and optimize MongoDB databases, including Replica Sets and Sharded Clusters.
- Develop efficient BSON document models while following MongoDB best practices.
- Build optimized indexes (Compound,



Partial, and Text Indexes) to improve query performance.
- Develop complex aggregation pipelines and perform query optimization.

Performance Optimization & Modernization

- Profile and optimize end-to-end data flows across Kafka, Redis, and MongoDB.
- Identify and resolve system bottlenecks including slow queries, consumer lag, caching issues, and storage inefficiencies.
- Implement modern data engineering best practices for scalable and reliable distributed systems.
- Participate in database modernization and system optimization initiatives.

Required Skills

Technical Skills

- 25 years of experience in backend or data platform engineering.
- Strong hands-on experience with:

- Apache Kafka

- Redis

- MongoDB

- Good understanding of

- Distributed Systems
- Event-Driven Architecture

- Data Streaming

- Caching Strategies

- NoSQL Database Design
- Experience with Kafka Consumer Groups, Partitions, Schema Registry, and Message Delivery semantics.
- Knowledge of MongoDB Aggregation Framework, Indexing, Replication, and Sharding.
- Familiarity with Redis Cluster, Sentinel, Pub/Sub, and advanced Redis data structures.
- Understanding of Linux environments, REST APIs, and monitoring tools.
- Experience with Git and Agile development methodologies.

Preferred Qualifications

- Exposure to enterprise-scale database modernization or migration projects.
- Knowledge of Docker, Kubernetes, CI/CD pipelines, or cloud platforms (AWS, Azure, or GCP) is an added advantage.
- Familiarity with monitoring and observability tools such as Prometheus, Grafana, or ELK Stack.

📌 Kafka Engineer (Thane)
🏢 3i Infotech
📍 Thane