Key Responsibilities
Design and develop real-time data pipelines using
Kafka, Spark, AWS Kinesis
, and related technologies.
Build and support scalable data processing solutions on
AWS
.
Modernize legacy ingestion frameworks using cloud-native services.
Develop and optimize large-scale distributed data processing systems.
Troubleshoot and support production data platforms and streaming applications.
Collaborate with cross-functional teams on enterprise data integration initiatives.

Required Skills & Experience
10+ years of experience in Data Engineering, Software Engineering, or related fields.
Solid expertise in
AWS Cloud Services
and cloud-native architectures.
Hands-on experience with
Apache Spark (Scala & PySpark)
.
Robust knowledge of the
Hadoop Ecosystem
and distributed computing.
Experience with
Kafka
, streaming technologies, and real-time data pipelines.
Experience with
AWS Kinesis, Kinesis Firehose, Flume
, or similar ingestion platforms.
Solid programming skills in
Python, Scala, and Java
.
Experience with
Big Data, Enterprise Data Warehousing, and Distributed Systems
.

📌 Sr Data Engineer Hyderabad (India)
🏢 Insight Global
📍 India