21 Nov
|
Arminus
|
Hyderabad
Mid-Level Big Data Engineer Location: Hyderabad
Employment Type: Full Time
Working Mode: Hybrid ( Wednessday & Thursday )
Experience Required: 6+ Years Open Positions - 3 Role Overview We are looking for an experienced Big Data Engineer to design, develop, test, and support cloud-based big data applications on AWS. The role involves building batch-processing solutions and API-based services, developing REST APIs and SOAP web services, and improving the performance and reliability of data-processing workloads. Key Responsibilities Big Data Applications: Analyze, design, develop, implement, and test big data applications covering batch processing and API-based workloads on AWS. API & Web Services: Develop and maintain REST APIs and SOAP web services using Scala, Java, and related data-processing technologies. AWS Solutions: Build and support AWS-based data-processing applications and microservices using EMR, EC2, ECS, S3, Airflow, Step Functions, and AWS API Gateway. Performance Tuning: Monitor, troubleshoot, and tune Spark, Hadoop, EMR, Java, and Python workloads to improve performance and reliability. Linux & Scripting: Use Linux, shell scripting, and Python to support development, automation, troubleshooting, and operational activities. Collaboration: Work with team members to deliver maintainable, reliable, and scalable data solutions. Experience & Qualifications Experience: 6+ years in analysis, coding, implementation, and testing of big data applications on AWS, including batch and API-based solutions. Programming:
Hands-on experience with Scala, PySpark, and Java; working knowledge of Python. APIs & Services: Experience developing REST APIs and SOAP web services. AWS: Practical experience with EMR, EC2, ECS, S3, Airflow, Step Functions, and AWS API Gateway. Big Data: Experience with Spark/Hadoop/EMR performance tuning; working knowledge of Hadoop, HDFS, MapReduce, and Cassandra. Systems: Linux experience, including shell scripting. M&S; Consulting | Mid-Level Big Data Engineer Page 1 of 2 Core Skills Required Must-Have Skills Valuable-to-Have Skills Scala PySpark Java Python Apache Spark Hadoop / HDFS MapReduce Cassandra AWS EMR / EC2 / ECS S3 Airflow Step Functions API Gateway REST APIs SOAP Linux Shell scripting Performance tuning Microservices DevOps CI/CD pipelines Maven Gradle Jenkins Artifactory AWS CodeCommit CloudFormation Git GitHub Bitbucket GitLab IntelliJ IDEA PyCharm Tools & Platforms Platforms & Tools AWS Amazon EMR Amazon EC2 Amazon ECS Amazon S3 AWS API Gateway Spark Hadoop Airflow Step Functions Cassandra Git GitHub Bitbucket GitLab CodeCommit IntelliJ IDEA PyCharm Maven Gradle Jenkins Artifactory CloudFormation Preferred Qualifications DevOps & Version Control: Familiarity with Git, GitHub, Bitbucket, GitLab, IntelliJ IDEA, or PyCharm. CI/CD: Experience creating CI/CD pipelines using Maven, Gradle, Jenkins, Artifactory, AWS CodeCommit, or CloudFormation. Role Details Experience 6+ years Location Hyderabad, India Work Model Hybrid Employment Type Full-Time
📌 Mid-Level Big Data Engineer (Hyderabad)
🏢 Arminus
📍 Hyderabad