Key Responsibilities

Apache Kafka (Event Streaming & Messaging)

- Design, implement, and maintain high-throughput Kafka topics, partitions, and consumer groups.
- Configure Kafka producers and consumers for optimal performance and reliability.
- Ensure message delivery guarantees and manage schema evolution using Confluent Schema Registry.
- Monitor and troubleshoot Kafka clusters, consumer lag, and broker performance.

Redis (Distributed Caching & In-Memory Data Store)

- Design and implement efficient caching strategies to improve application performance and reduce database load.
- Utilize Redis data structures such as Hashes, Lists, Sets, Sorted Sets, HyperLogLogs, and Geospatial indexes.
- Configure and manage Redis Clusters, Sentinel deployments, Pub/Sub messaging, and persistence mechanisms.
- Optimize cache performance and resolve issues such as cache stampedes and memory fragmentation.

MongoDB (NoSQL Database)

- Design, implement, and optimize MongoDB databases, including Replica Sets and Sharded Clusters.
- Develop efficient BSON document models while following MongoDB best practices.
- Build optimized indexes (Compound, Partial, and Text Indexes) to improve query performance.
- Develop complex aggregation pipelines and perform query optimization.

Performance Optimization & Modernization

- Profile and optimize end-to-end data flows across Kafka, Redis, and MongoDB.




- Identify and resolve system bottlenecks including slow queries, consumer lag, caching issues, and storage inefficiencies.
- Implement contemporary data engineering best practices for scalable and reliable distributed systems.
- Participate in database modernization and system optimization initiatives.

Required Skills

Technical Skills

- 2–3 years of experience in backend or data platform engineering.
- Strong hands-on experience with:
- Apache Kafka
- Redis
- MongoDB

- Good understanding of:

- Distributed Systems
- Event-Driven Architecture
- Data Streaming
- Caching Strategies
- NoSQL Database Design

- Experience with Kafka Consumer Groups, Partitions, Schema Registry, and Message Delivery semantics.
- Knowledge of MongoDB Aggregation Framework, Indexing, Replication, and Sharding.
- Familiarity with Redis Cluster, Sentinel, Pub/Sub, and advanced Redis data structures.
- Understanding of Linux environments, REST APIs, and monitoring tools.
- Experience with Git and Agile development methodologies.

Preferred Qualifications

- Exposure to enterprise-scale database modernization or migration projects.
- Knowledge of Docker, Kubernetes, CI/CD pipelines, or cloud platforms (AWS, Azure, or GCP) is an added advantage.
- Familiarity with monitoring and observability tools such as Prometheus, Grafana, or ELK Stack.

📌 Kafka Engineer (Maharashtra)
🏢 3i Infotech
📍 Maharashtra