Responsibilities and Duties: · • Design & develop data pipelines for real-time and batch data ingestion and processing using Confluent Kafka, ksqlDB, Kafka Connect, and Apache Flink. • Build and configure Kafka Connectors to ingest data from various sources (databases, APIs, message queues, etc.) into Kafka. • Develop Flink applications for complex event processing, stream enrichment, and real-time analytics. • Develop and optimize ksqlDB queries for real-time data transformations, aggregations, and filtering. • Implement data quality checks and monitoring to ensure data accuracy and reliability throughout the pipeline. • Monitor and troubleshoot data pipeline performance, identify bottlenecks, and implement optimizations. • Automate data pipeline deployment, monitoring, and maintenance tasks. • Stay up-to-date with the latest advancements in data streaming technologies and best practices. • Contribute to the development of data engineering standards and best practices within the organization.



• Participate in code reviews and contribute to a collaborative and supportive team environment. • Work closely with other architects and tech leads in India & US and create POCs and MVPs • Provide regular updates on the tasks, status and risks to project manager Requirements: • Bachelor's degree or higher from a reputed university • 8 to 10 years total experience with majority of that experience related to ETL/ELT, big data, Kafka etc. • Proficiency in developing Flink applications for stream processing and real-time analytics. • Solid understanding of data streaming concepts and architectures. • Extensive experience with Confluent Kafka, including Kafka Brokers, Producers, Consumers, and Schema Registry. • Hands-on experience with ksqlDB for real-time data transformations and stream processing. • Experience with Kafka Connect and building custom connectors. • Extensive experience in implementing large scale data ingestion and curation solutions • Good hands on experience in

📌 Kafka_Lead_Citius (Chennai)
🏢 TalentOla
📍 Chennai