Key Responsibilities

- Design, develop, and maintain high-performance real-time data processing applications using Apache Flink.

- Develop in-flight transformation logic leveraging Apache Flink, Kafka Streams, KSQLDB, and Kafka Single Message Transforms (SMTs).

- Build and maintain scalable mapping frameworks for transforming source system data into harmonized enterprise data models.

- Implement complex event processing, enrichment, filtering, aggregation, and windowing logic for streaming workloads.

- Optimize streaming pipeline performance, ensuring low latency, high throughput, and efficient resource utilization.

- Design solutions to handle late-arriving and out-of-order events using event-time processing, watermarks, checkpointing, and state management.

- Integrate streaming applications with Kafka, Schema Registry, APIs, and downstream analytical platforms.

- Implement robust error handling, retry mechanisms, dead-letter queues (DLQ), and monitoring for streaming applications.

- Collaborate with Solution Architects, Data Architects, API teams, and business stakeholders to define transformation rules and streaming integration patterns.

- Develop automated testing frameworks and CI/CD pipelines for streaming applications.

- Monitor production streaming jobs and troubleshoot performance, scalability, and reliability issues.

Required Skills & Experience

- 610+ years of experience in Data Engineering with strong expertise in real-time streaming solutions.





- Hands-on experience with Apache Flink for enterprise-scale stream processing.

- Strong knowledge of Apache Kafka, Kafka Streams, KSQLDB, Kafka Connect, and Single Message Transforms (SMTs).

- Experience designing event-driven data architectures and real-time data integration pipelines.

- Strong understanding of event-time processing, watermarks, windowing, checkpointing, stateful stream processing, and fault tolerance.

- Experience developing data transformation and harmonization frameworks for enterprise data platforms.

- Proficiency in Java and/or Scala; Python knowledge is desirable.

- Experience with Avro, Protobuf, JSON, Schema Registry, and serialization techniques.

- Strong SQL skills and experience working with large-scale structured and semi-structured datasets.

- Experience with cloud platforms such as Azure, AWS, or GCP is preferred.

- Familiarity with CI/CD, Git, Docker, Kubernetes, and monitoring tools is an advantage.

Preferred Qualifications

- Experience with Confluent Platform and enterprise Kafka deployments.

- Exposure to contemporary data lakehouse platforms such as Databricks, Snowflake, or Delta Lake.

- Knowledge of CDC patterns, API integration, and enterprise integration architectures.

- Experience working in Agile/Scrum delivery environments.

- Excellent analytical, problem-solving, and stakeholder communication skills.

📌 Apache Flink Transformation Engineer (Bengaluru)
🏢 Genpact
📍 Bengaluru