- Design, develop, test, and deploy scalable data pipelines using Kafka on AWS cloud platform.
- Collaborate with cross-functional teams to identify business requirements and design solutions that meet those needs.
- Develop automated testing frameworks using Python/Java to ensure high-quality code delivery.
- Troubleshoot issues related to data processing workflows and provide recommendations for improvement.
Job Requirements :
- 4-8 years of experience in big data engineering with expertise in Kafka clustering.
- Migration experience
- Solid understanding of Java programming language and its applications in big data processing.
- Experience working with AWS cloud platform including S3 buckets, Lambda functions, and EMR clusters.
- Proficiency in developing automated tests using Python or similar languages.