07 Sep
|
Zorba Consulting India
|
Pune
07 Sep
Zorba Consulting India
Pune
We are looking for an experienced Data Streaming Engineer with strong hands-on expertise in Apache Flink and Apache Spark to design, develop, and optimize large-scale real-time and batch data processing solutions.
The candidate should have strong experience in Flink DataStream API, Flink Table API/SQL, Spark Core, Spark SQL, and Spark Structured Streaming, with a solid understanding of distributed stream processing and high-volume data pipelines.
Mandatory Skills
8–10 years of experience in Data Engineering / Big Data / Streaming.
Strong hands-on experience with Apache Flink.
Expertise in Flink DataStream API.
Solid knowledge of Flink Table API & Flink SQL API.
Experience with stateful stream processing, event-time processing, windowing, watermarks, and late-event handling.
Strong understanding of Flink checkpointing, fault tolerance, and distributed deployment.
Robust experience with Apache Spark.
Hands-on expertise in Spark Core, Spark SQL, and Spark Structured Streaming.
Experience developing high-volume ETL/data transformation pipelines.
Strong understanding of Spark execution architecture, resource management, performance tuning, and optimization.
Solid understanding of distributed systems and real-time data processing.
Proficiency in at least one programming language: Java / Scala / Python.
Good to Have
Apache Kafka / Azure Event Hubs or other messaging platforms.
Azure / AWS / GCP cloud experience.
Docker / Kubernetes.
CI/CD and DevOps practices.
Experience with Data Lakes / Lakehouse architectures.
Microservices and event-driven architecture.
Monitoring, logging, troubleshooting, and data quality.
Key Responsibilities
Design and develop scalable real-time streaming pipelines using Flink and Spark.
Build event-driven and near-real-time data processing solutions.
Develop batch and streaming ETL workflows.
Optimize pipelines for low latency, high throughput, reliability, and scalability.
Implement Flink stat
📌 Data Streaming Engineer – Apache Flink Pune
🏢 Zorba Consulting India
📍 Pune