- Experience: 4 6 years (Mid-level)
- Location: Hyderabad (Onsite on Wednesdays and Thursdays)
- Required Skills: Python, hands-on AWS (EC2, Lambda, and mandatory EMR), and EMR performance and cost optimization.
- Budget: We can go a little higher on the budget for this role.

Interview Process:

- L1: Technical Engineer (Virtual)
- L2: SME (Virtual)
- L3: Engineering Manager (In-person)
Overview

Location: Hybrid Hyderabad

Employment Type: full time

Experience: 5 - 6 years

Compensation: INR 1,153,440 - 1,441,800

Experience Required

Extracted: 5+ years of hands-on experience developing enterprise-scale applications, data platforms, or distributed systems. 5+ years of experience developing, and operating Big Data platforms and cloud-based infrastructure services, preferably using AWS EMR and the Hadoop ecosystem.

Overview

The role focuses on building scalable data engineering and backend services for large-scale batch Big Data applications. You will develop and maintain data transformation pipelines using Spark/SQL/Hive and build cloud-native solutions on AWS. The position also involves workflow orchestration with Apache Airflow, production troubleshooting, and collaboration in an Agile/Scrum environment.

Key Responsibilities

Develop API-driven systems to govern, manage, and monitor large-scale batch Big Data applications

Build scalable backend services and data engineering solutions supporting data processing and operational workflows

Develop and maintain data transformation processes using Spark, SQL, Hive, Python, Scala, and related technologies

Build cloud-native data and backend solutions using AWS services (S3, EC2, EMR, Lambda, DynamoDB, API Gateway)

Build and enhance workflow orchestration using Apache Airflow (advanced DAG design, dependency management,



scheduling, monitoring, failure handling)

Define technical scope/objectives and implementation approaches via requirements gathering, technical research, and process definition

Participate in architecture reviews, code reviews, performance tuning, and operational readiness activities

Contribute to test planning and validation for integrations, functional areas, and deliverables

Partner with cross-functional teams (product owners, data engineers, backend engineers, QA, DevOps) in an Agile/Scrum environment

Troubleshoot production issues, improve reliability, and drive automation across data and backend workflows

Required Qualifications

5+ years of hands-on experience developing enterprise-scale applications, data platforms, or distributed systems

5+ years of experience developing and operating Big Data platforms and cloud-based infrastructure services (preferably AWS EMR and Hadoop ecosystem)

Hands-on experience with AWS, Spark, Python and/or Scala, Airflow, SQL, Hive, and related data processing technologies

Proficiency in either Python or Scala to build production-grade data pipelines and backend services

Experience with AWS services: S3, EC2, EMR, Lambda, DynamoDB, API Gateway

Strong experience with Apache Airflow orchestration (complex workflows, scheduling, dependency handling, monitoring, operational support)

Strong understanding of data engineering concepts (data pipelines,



data transformation, batch processing, real-time processing, data quality, performance optimization)

Excellent analytical and problem-solving skills

Robust communication and presentation skills for technical and non-technical audiences

Experience with version control and CI/CD tools including Git and Jenkins

Ability to work effectively in cross-functional Agile/Scrum teams

Technical Skills Required

AWS

Apache Spark

Python

Scala

Apache Airflow

SQL

Hive

AWS EMR

Hadoop ecosystem

Git

Jenkins

CI/CD

Data pipelines

Data transformation

Batch processing

Real-time processing

Data quality

Performance optimization

Nice-to-have

LLMs

Generative AI

Agentic AI

AI-assisted engineering workflows

API design

Microservices

Event-driven architecture

Serverless backend development

Infrastructure as code

Automated testing

Data governance

Metadata management

Lineage

Data observability

Operational controls

Tools/Platforms

AWS S3

AWS EC2

AWS EMR

AWS Lambda

Amazon DynamoDB

Amazon API Gateway

Apache Airflow

Apache Spark

Hive

Git

Jenkins

Preferred Qualifications

Bachelor's degree in Computer Science, Engineering, Mathematics, Information Systems, or a related technical field, or equivalent practical experience

Experience with LLMs, Generative AI, Agentic AI, or AI-assisted engineering workflows

Experience with API design, microservices, event-driven architecture, or serverless backend development

Experience with CI/CD pipelines, infrastructure as code, automated testing, and production deployment practices

Experience with data governance, metadata management, lineage, data observability, or operational controls for enterprise data platforms

📌 Mid-level Data Engineer - 5015_4-6yrs_Hyderabad (Hyderabad)
🏢 Arminus
📍 Hyderabad