Job Role: Senior Data Engineer

Experience: 8+ Years

Location: Pune

Key Responsibilities
Design and develop scalable ETL and data processing pipelines on AWS.
Build batch and real-time data pipelines using

PySpark, Spark Streaming, and Kafka

.
Develop and optimize Data Lake solutions using

Apache Hudi or

Apache Iceberg

.
Build and manage data processing workflows using

AWS Glue and

Amazon EMR

.
Automate infrastructure deployment using

Terraform and containerize applications using

Docker and

Amazon ECS

.
Optimize large-scale distributed data processing for performance, scalability, and reliability.
Implement data cataloging and metadata management using

AWS Glue Data Catalog

.
Collaborate with cross-functional teams to deliver secure, high-quality data engineering solutions.




Troubleshoot production issues and continuously improve data platform performance.

Key Skills
8+ years of experience in Data Engineering.
Solid expertise in

AWS Glue

,

PySpark

,

Amazon EMR

, and

Spark Streaming

.
Hands-on experience with

Apache Kafka for real-time data streaming.
Experience with

Apache Hudi or

Apache Iceberg

.
Robust programming skills in

Python and Object-Oriented Programming (OOPS).
Experience with

AWS Glue ETL and

Glue Data Catalog

.
Hands-on experience with

Docker

,

Amazon ECS

, and

Terraform

.

📌 Senior Data Engineer Pune
🏢 TekPillar®
📍 Pune