06 Sep
|
Zorba Consulting India
|
Pune
06 Sep
Zorba Consulting India
Pune
We are looking for an experienced Data Streaming Engineer with strong hands-on expertise in Apache Flink and Apache Spark to design, develop, and optimize large-scale real-time and batch data processing solutions.
The candidate should have strong experience in Flink DataStream API, Flink Table API/SQL, Spark Core, Spark SQL, and Spark Structured Streaming, with a solid understanding of distributed stream processing and high-volume data pipelines.
Mandatory Skills
- 8–10 years of experience in Data Engineering / Big Data / Streaming.
- Strong hands-on experience with Apache Flink.
- Expertise in Flink DataStream API.
- Strong knowledge of Flink Table API & Flink SQL API.
- Experience with stateful stream processing, event-time processing, windowing, watermarks, and late-event handling.
- Strong understanding of Flink checkpointing, fault tolerance, and distributed deployment.
- Robust experience with Apache Spark.
- Hands-on expertise in Spark Core, Spark SQL, and Spark Structured Streaming.
- Experience developing high-volume ETL/data transformation pipelines.
- Strong understanding of Spark execution architecture, resource management, performance tuning, and optimization.
- Strong understanding of distributed systems and real-time data processing.
- Proficiency in at least one programming language: Java / Scala / Python.
Good to Have
- Apache Kafka / Azure Event Hubs or other messaging platforms.
- Azure / AWS / GCP cloud experience.
- Docker / Kubernetes.
- CI/CD and DevOps practices.
- Experience with Data Lakes / Lakehouse architectures.
- Microservices and event-driven architecture.
- Monitoring, logging, troubleshooting, and data quality.
Key Responsibilities
- Design and develop scalable real-time streaming pipelines using Flink and Spark.
- Build event-driven and near-real-time data processing solutions.
- Develop batch and streaming ETL workflows.
- Optimize pipelines for low latency, high throughput, reliability, and scalability.
- Implement Flink stat
📌 Data Streaming Engineer – Apache Flink (Pune)
🏢 Zorba Consulting India
📍 Pune