Mid-Level Big Data Engineer
Location: Hyderabad
Employment Type: Full Time
Working Mode: Hybrid ( Wednessday & Thursday )
Experience Required: 6+ Years
Open Positions - 3
Role Overview We are looking for an experienced Big Data Engineer to design, develop, test, and support cloud-based big data applications on AWS. The role involves building batch-processing solutions and API-based services, developing REST APIs and SOAP web services, and improving the performance and reliability of data-processing workloads. Key Responsibilities Big Data Applications: Analyze, design, develop, implement, and test big data applications covering batch processing and API-based workloads on AWS. API & Web Services: Develop and maintain REST APIs and SOAP web services using Scala, Java, and related data-processing technologies. AWS Solutions: Build and support AWS-based data-processing applications and microservices using EMR, EC2, ECS, S3, Airflow, Step Functions, and AWS API Gateway. Performance Tuning: Monitor, troubleshoot, and tune Spark, Hadoop, EMR, Java, and Python workloads to improve performance and reliability. Linux & Scripting: Use Linux, shell scripting, and Python to support development, automation, troubleshooting, and operational activities. Collaboration: Work with team members to deliver maintainable, reliable, and scalable data solutions. Experience & Qualifications Experience: 6+ years in analysis, coding, implementation, and testing of big data applications on AWS, including batch and API-based solutions. Programming:
Hands-on experience with Scala, PySpark, and Java; working knowledge of Python. APIs & Services: Experience developing REST APIs and SOAP web services. AWS: Practical experience with EMR, EC2, ECS, S3, Airflow, Step Functions, and AWS API Gateway. Big Data: Experience with Spark/Hadoop/EMR performance tuning; working knowledge of Hadoop, HDFS, Map Reduce, and Cassandra. Systems: Linux experience, including shell scripting. M&S; Consulting | Mid-Level Big Data Engineer Page 1 of 2 Core Skills Required Must-Have Skills Good-to-Have Skills Scala PySpark Java Python Apache Spark Hadoop / HDFS Map Reduce Cassandra AWS EMR / EC2 / ECS S3 Airflow Step Functions API Gateway REST APIs SOAP Linux Shell scripting Performance tuning Microservices Dev Ops CI/CD pipelines Maven Gradle Jenkins Artifactory AWS Code Commit Cloud Formation Git Git Hub Bitbucket Git Lab IntelliJ IDEA PyCharm Tools & Platforms Platforms & Tools AWS Amazon EMR Amazon EC2 Amazon ECS Amazon S3 AWS API Gateway Spark Hadoop Airflow Step Functions Cassandra Git Git Hub Bitbucket Git Lab Code Commit IntelliJ IDEA PyCharm Maven Gradle Jenkins Artifactory Cloud Formation Preferred Qualifications Dev Ops & Version Control: Familiarity with Git, Git Hub, Bitbucket, Git Lab, IntelliJ IDEA, or PyCharm. CI/CD: Experience creating CI/CD pipelines using Maven, Gradle, Jenkins, Artifactory, AWS Code Commit, or Cloud Formation. Role Details Experience 6+ years Location Hyderabad, India Work Model Hybrid Employment Type Full time
📌 Mid-Level Big Data Engineer (India)
🏢 Arminus
📍 India