Job Description
Understand the design specifications.
Follow the best practices for core product development.
​Data ingestion for distributed system with parallel processing, in Java, CPP or Golang for
from cloud storage – Amazon S3/Azure Cloud Storage/Google Cloud Storage, relational database – Snowflake, Google BigQuery, PostgreSQL, Files, Kafka, data lakehouse (Iceberg)
High-availability (HA) loading, cross-region replication
Loading monitoring/error reporting
Spark Connector
Third-party management – Kafka, Kafka Connect

Requirements
building distributed systems applying applying parallel processing.
Any or of the following – Kafka, Zookeeper, Spark, Stream processing.
Experience in Kafka Connect, Kafka stream, Kafka security and customization
Spark connectors
Event driven architectures.
Agile development
CICD

​
Nice to have:
· gRPC protocol experience and multi-threading
· Zookeeper/ ETCD/ Consul
· Familiarity with distributed consensus algorithms (Paxos / Raft)
· Docker, Kubernetes
​

📌 Principal/senior Data Engineer Hyderabad
🏢 Agivant Technologies
📍 Hyderabad