Key Responsibilities

- Design and develop real-time data pipelines using

Kafka, Spark, AWS Kinesis

, and related technologies.

- Build and support scalable data processing solutions on

AWS

.

- Modernize legacy ingestion frameworks using cloud-native services.

- Develop and optimize large-scale distributed data processing systems.

- Troubleshoot and support production data platforms and streaming applications.

- Collaborate with cross-functional teams on enterprise data integration initiatives.

Required Skills & Experience

- 10+ years of experience in Data Engineering, Software Engineering, or related fields.

- Robust expertise in

AWS Cloud Services and cloud-native architectures.

- Hands-on experience with

Apache Spark (Scala & PySpark)

.

- Strong knowledge of the

Hadoop Ecosystem and distributed computing.

- Experience with

Kafka

, streaming technologies, and real-time data pipelines.

- Experience with

AWS Kinesis, Kinesis Firehose, Flume

, or similar ingestion platforms.

- Strong programming skills in

Python, Scala, and Java

.

- Experience with

Big Data, Enterprise Data Warehousing, and Distributed Systems

.

📌 Sr. Data Engineer (Hyderabad)
🏢 Insight Global
📍 Hyderabad