Senior Data Engineer Kafka and Streaming Data Processing (Kolkata)

Senior Data Engineer Kafka and Streaming Data Processing (Kolkata)

17 Nov
|
Nihilent
|
Kolkata

17 Nov

Nihilent

Kolkata

Senior Data Engineer Kafka & Streaming Data Processing
Location: Pune

Experience: 7-12 Years

About the Role:
We're hiring a Senior Data Engineer to build and scale real-time data processing pipelines using Apache Kafka and Debezium on AWS and open source stack like Apache Iceberg, Open metadata, Atlas, Apache Airflow. If you're passionate about streaming architectures and enjoy working across the AWS data ecosystem and open source stack, this role is for you.

Responsibilities:
Design & manage Kafka-based streaming pipelines (topics, partitioning, consumer groups), Flink

Design & manage Spark based Batch pipelines using Spark based framework

Configure and maintain Debezium connectors for real-time CDC
Build scalable ETL/ELT workflows using PySpark and AWS Airflow





Solid experience in utilizing Iceberg tables and AWS open source system.

Manage and optimize AWS Redshift for analytics & reporting
Ensure pipeline reliability, data quality, and performance at scale
Collaborate with cross-functional teams on data requirements

What We're Looking For:
10-12 years of Data Engineering experience with strong hands-on Kafka, CDC, AWS skills (must-have)
Experience with Debezium / CDC tools
Proficiency in PySpark, SQL, AWS Airflow, Glue, Iceberg and Redshift
Working knowledge of AWS services (S3, IAM, Lambda)
Strong SQL and data modeling fundamentals

Nice to Have:
Experience in the Finance/Banking domain

📌 Senior Data Engineer Kafka and Streaming Data Processing (Kolkata)
🏢 Nihilent
📍 Kolkata

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: senior data engineer kafka and streaming data processing (kolkata) / kolkata

Subscribe to this job alert:

Get the latest job offers by email for: senior data engineer kafka and streaming data processing (kolkata) / kolkata