Roles and Responsibilities :
- Design, develop, test, deploy and maintain scalable data pipelines using Hadoop ecosystem (Hive) on Kafka messaging queues.
- Collaborate with cross-functional teams to identify business requirements and design solutions that meet those needs.
- Develop automated deployment scripts using Docker containers for efficient infrastructure management.
- Troubleshoot complex issues related to data processing workflows and optimize system performance.
Job Requirements :
- 5-9 years of experience in software development with expertise in Python programming language.
- Robust understanding of JavaScript frameworks such as Node.js or similar technologies.
- Experience working with distributed computing systems like Apache Spark or Hadoop Distributed File System (HDFS).
- Proficiency in managing message queues like Kafka for real-time data processing.