Role OverviewWe are seeking a Senior Data Engineer with deep expertise in Scala and Apache Kafka. You will design, build, and maintain high-throughput, fault-tolerant event-driven microservices and data pipelines.
Key Responsibilities
- Design distributed systems: Build scalable stream-processing applications using Scala and Kafka.
- Develop event pipelines: Implement real-time data ingestion, processing, and distribution layers.
- Optimize performance: Tune Kafka clusters, topics, partitions, and Scala applications for low latency.
- Maintain code quality: Write clean, functional, testable, and highly concurrent Scala code.
- Integrate ecosystems: Connect Kafka with databases, data lakes, and external APIs.
- Monitor and troubleshoot: Resolve production bottlenecks, data lag,
and cluster connectivity issues.
Technical Requirements
- Language: Strong proficiency in Scala (Functional Programming paradigms).
- Streaming: Deep knowledge of Apache Kafka, Kafka Connect, and Kafka Streams.
- Frameworks: Experience with Akka/Pekko, ZIO, Cats Effect, or Spark Streaming.
- Data Layers: Familiarity with Avro, Protobuf, JSON serialization, and Schema Registry.
- Cloud & DevOps: Experience with AWS/GCP, Docker, Kubernetes, and CI/CD pipelines.
Soft Skills
- Robust problem-solving abilities.
- Excellent technical communication.
- Collaboration in Agile teams.
📌 Data Engineer (Pune)
🏢 LTM
📍 Pune