Responsibilities and Duties: · • Design & develop data pipelines for real-time and batch data ingestion and processing using Confluent Kafka, ksqlDB, Kafka Connect, and Apache Flink.

- Build and configure Kafka Connectors to ingest data from various sources (databases, APIs, message queues, etc.) into Kafka.
- Develop Flink applications for complex event processing, stream enrichment, and real-time analytics.
- Develop and optimize ksqlDB queries for real-time data transformations, aggregations, and filtering.
- Implement data quality checks and monitoring to ensure data accuracy and reliability throughout the pipeline.
- Monitor and troubleshoot data pipeline performance, identify bottlenecks, and implement optimizations.
- Automate data pipeline deployment, monitoring, and maintenance tasks.
- Stay up-to-date with the latest advancements in data streaming technologies and best practices.
- Contribute to the development of data engineering standards and best practices within the organization.
- Participate in code reviews and contribute to a collaborative and supportive team workplace.




- Work closely with other architects and tech leads in India & US and create POCs and MVPs • Provide regular updates on the tasks, status and risks to project manager Requirements:
- Bachelor's degree or higher from a reputed university • 8 to 10 years total experience with majority of that experience related to ETL/ELT, big data, Kafka etc.
- Proficiency in developing Flink applications for stream processing and real-time analytics.
- Strong understanding of data streaming concepts and architectures.
- Extensive experience with Confluent Kafka, including Kafka Brokers, Producers, Consumers, and Schema Registry.
- Hands-on experience with ksqlDB for real-time data transformations and stream processing.
- Experience with Kafka Connect and building custom connectors.
- Extensive experience in implementing large scale data ingestion and curation solutions • Good hands on experience in big data technology stack with any cloud platform - • Excellent problem-solving, analytical, and communication skills.
- Ability to work independently and as part of a team

📌 Kafka_Lead_Citius (Chennai)
🏢 TalentOla
📍 Chennai