Key Responsibilities

Design and develop real-time data pipelines using Kafka, Spark, AWS Kinesis , and related technologies.

Build and support scalable data processing solutions on AWS .

Modernize legacy ingestion frameworks using cloud-native services.

Develop and optimize large-scale distributed data processing systems.

Troubleshoot and support production data platforms and streaming applications.

Collaborate with cross-functional teams on enterprise data integration initiatives.

Required Skills & Experience

- 10+ years of experience in Data Engineering, Software Engineering, or related fields.
- Robust expertise in AWS Cloud Services and cloud-native architectures.
- Hands-on experience with Apache Spark (Scala & PySpark) .
- Strong knowledge of the Hadoop Ecosystem and distributed computing.
- Experience with Kafka , streaming technologies, and real-time data pipelines.
- Experience with AWS Kinesis, Kinesis Firehose, Flume , or similar ingestion platforms.
- Strong programming skills in Python, Scala, and Java .
- Experience with Big Data, Enterprise Data Warehousing, and Distributed Systems .

📌 Senior Data Engineer : Big Data (Secunderabad)
🏢 Insight Global
📍 Secunderabad