Job Title: Kafka flink Engineer

Experience: 6+ Years
Location: Remote
Shift Timings: 2:00 PM – 11:00 PM IST

Role Overview

We are seeking an experienced Confluent Flink Engineer with strong expertise in real-time data streaming, event-driven architectures, and large-scale distributed systems. The ideal candidate should have hands-on experience with Apache Flink, Confluent Kafka, Kafka Streams, and streaming ETL pipelines, along with a strong understanding of cloud-native data processing solutions.

The role involves designing, developing, and optimizing real-time streaming applications that process high-volume data efficiently while ensuring scalability, fault tolerance, and performance.

Key ResponsibilitiesStream Processing & Data Engineering

- Design, develop, and maintain real-time data processing applications using Apache Flink.
- Build and manage large-scale streaming ETL pipelines.
- Develop event-driven architectures using Confluent Kafka and Kafka Streams.
- Implement real-time data transformation, aggregation, enrichment, and analytics workflows.

Kafka & Confluent Platform

- Configure and manage Confluent Kafka environments.
- Work with Kafka Connect, Schema Registry, and other Confluent Platform components.
- Design efficient topic structures, partitioning strategies, and consumer group architectures.
- Ensure reliable event processing and message delivery.

Performance & Scalability

- Optimize Flink jobs for performance, throughput, and resource utilization.
- Implement checkpointing, state management, and fault-tolerant processing strategies.
- Monitor and troubleshoot streaming applications and data pipelines.
- Ensure scalability and high availability of streaming platforms.

Cloud & DevOps





- Deploy and manage streaming workloads on AWS, Azure, or GCP.
- Participate in CI/CD pipeline implementation and automation.
- Collaborate with DevOps teams to improve deployment, monitoring, and operational practices.

Collaboration

- Work closely with Data Engineers, Architects, Product Teams, and Business Stakeholders.
- Participate in architecture discussions and technical design reviews.
- Contribute to best practices, documentation, and knowledge sharing.

Required Skills & Experience

- 6+ years of software engineering or data engineering experience.
- Strong hands-on experience with Apache Flink.
- Extensive experience with Confluent Kafka and Kafka Streams.
- Strong understanding of real-time streaming and event-driven architectures.
- Experience with distributed systems and large-scale data processing.
- Proficiency in Java, Scala, or Python.
- Experience building and managing streaming ETL pipelines.
- Strong knowledge of:
- Kafka Connect
- Schema Registry
- Confluent Platform Components

- Strong understanding of:

- Data Partitioning
- State Management
- Checkpointing
- Fault Tolerance

- Experience with AWS, Azure, or GCP cloud platforms.
- Familiarity with CI/CD pipelines and DevOps practices.

Preferred Skills

- Experience with Kubernetes and containerized deployments.
- Knowledge of data lake and up-to-date data platform architectures.
- Exposure to monitoring tools such as Prometheus, Grafana, or Datadog.
- Experience with Spark Streaming or other stream processing technologies.

Key Competencies

- Real-Time Data Streaming
- Apache Flink Development
- Confluent Kafka & Kafka Streams
- Distributed Systems
- Event-Driven Architecture
- Streaming ETL Pipelines
- Cloud Data Engineering
- Performance Optimization
- CI/CD & DevOps

📌 Kafka Engineer (India)
🏢 kooe private
📍 India