Key Responsibilities
- Design, develop, implement, and maintain scalable Big Data applications and data processing pipelines on AWS Cloud.
- Build and support both API-based microservices and batch processing solutions using Java, Scala, Kotlin, and Python.
- Develop and optimize large-scale distributed data processing applications utilizing:
- Apache Spark
- Hadoop Ecosystem
- HDFS
- MapReduce
- Cassandra
- Create and maintain RESTful APIs and SOAP Web Services for enterprise data integration.
- Develop cloud-native solutions leveraging AWS services including:
- EMR
- EC2
- ECS
- S3
- Airflow
- AWS Step Functions
- AWS API Gateway
- Perform performance tuning and optimization of Spark, Hadoop, EMR, Java, and Python-based applications.
- Develop Linux-based automation scripts using Shell Scripting and Python.
- Collaborate with cross-functional teams including Data Engineering, Architecture, DevOps, and Product teams.
- Implement and maintain CI/CD pipelines to automate application deployment and infrastructure provisioning.
- Participate in code reviews, troubleshoot production issues,
and ensure adherence to engineering best practices.
Required Qualifications
Experience
- 7+ years of experience in software development, data engineering, and Big Data application development.
- Proven experience building scalable data platforms and distributed processing systems on AWS Cloud.