We are looking for an experienced Data Streaming Engineer with strong hands-on expertise in Apache Flink and Apache Spark to design, develop, and optimize large-scale real-time and batch data processing solutions.

The candidate should have strong experience in Flink DataStream API, Flink Table API/SQL, Spark Core, Spark SQL, and Spark Structured Streaming, with a solid understanding of distributed stream processing and high-volume data pipelines.

Mandatory Skills

- 8–10 years of experience in Data Engineering / Big Data / Streaming.
- Strong hands-on experience with Apache Flink.
- Expertise in Flink DataStream API.
- Strong knowledge of Flink Table API & Flink SQL API.
- Experience with stateful stream processing, event-time processing, windowing, watermarks, and late-event handling.
- Strong understanding of Flink checkpointing, fault tolerance, and distributed deployment.
- Strong experience with Apache Spark.
- Hands-on expertise in Spark Core, Spark SQL, and Spark Structured Streaming.
- Experience developing high-volume ETL/data transformation pipelines.
- Robust understanding of Spark execution architecture, resource management, performance tuning, and optimization.
- Strong understanding of distributed systems and real-time data processing.




- Proficiency in at least one programming language: Java / Scala / Python.

Good to Have
- Apache Kafka / Azure Event Hubs or other messaging platforms.
- Azure / AWS / GCP cloud experience.
- Docker / Kubernetes.
- CI/CD and DevOps practices.
- Experience with Data Lakes / Lakehouse architectures.
- Microservices and event-driven architecture.
- Monitoring, logging, troubleshooting, and data quality.

Key Responsibilities
- Design and develop scalable real-time streaming pipelines using Flink and Spark.
- Build event-driven and near-real-time data processing solutions.
- Develop batch and streaming ETL workflows.
- Optimize pipelines for low latency, high throughput, reliability, and scalability.
- Implement Flink state management, checkpointing, watermarks, and fault tolerance.
- Tune Spark jobs and optimize resource utilization.
- Troubleshoot production data processing and streaming issues.
- Implement monitoring, logging, data quality, governance, and security standards.
- Collaborate with Data Engineers, Architects, and business stakeholders.

Skills: apache spark,apache flink,kafka,sql

📌 Data Streaming Engineer – Apache Flink & Spark (Chennai)
🏢 Zorba AI
📍 Chennai