We are looking for an experienced AWS Big Data Engineer with strong expertise in Python/Scala, Apache Spark, AWS cloud services, and Apache Airflow. The ideal candidate will have hands-on experience developing scalable data engineering solutions, API-driven backend services, and distributed Big Data applications.

Key Responsibilities
Design, develop, test, and deploy scalable Big Data applications and data engineering solutions on AWS. Develop API-driven backend services to manage, monitor, and govern large-scale batch data processing applications. Build and maintain data pipelines using Apache Spark, Python, Scala, SQL, Hive, and Hadoop. Work with AWS services including S3, EC2, EMR, Lambda, DynamoDB, and API Gateway. Design and enhance Apache Airflow DAGs for workflow orchestration, scheduling, dependency management, monitoring, and failure handling. Develop scalable batch and real-time data processing solutions. Perform performance tuning, optimization, troubleshooting, and production support for distributed data applications. Participate in architecture reviews, code reviews, testing, and deployment activities. Collaborate with data engineers, backend developers, QA, DevOps, and product teams in an Agile/Scrum environment. Implement automation, CI/CD practices, and operational improvements to enhance system reliability. Mandatory Skills 5+ years of experience in Big Data Engineering, distributed systems, or enterprise application development.



Strong hands-on experience with AWS cloud services, particularly S3, EC2, EMR, Lambda, DynamoDB, and API Gateway. Robust programming experience in Python or Scala. Hands-on experience with Apache Spark, Hadoop, and Hive. Strong experience in Apache Airflow, including complex DAG development, scheduling, dependencies, monitoring, and error handling. Strong SQL and data engineering fundamentals. Experience developing APIs, backend services, or API-driven applications. Experience with Git, Jenkins, and CI/CD pipelines. Good understanding of batch processing, data transformation, data quality, and performance optimization. Experience working in Agile/Scrum environments. Good to Have Exposure to LLMs, Generative AI, Agentic AI, or AI-assisted engineering. Experience with REST APIs, microservices, event-driven architecture, and serverless applications. Knowledge of Infrastructure as Code and automated testing. Experience with data governance, metadata management, data lineage, and data observability. Bachelor's degree in Computer Science, Engineering, Mathematics, or a related technical discipline. Key Skills AWS | AWS EMR | Apache Spark | Python | Scala | Apache Airflow | Hadoop | Hive | SQL | S3 | EC2 | Lambda | DynamoDB | API Gateway | REST APIs | Microservices | Git | Jenkins | CI/CD

Skills:
git,aws,big data,rest apis,scala,sql,ci/cd,python,microservices

📌 AWS Big Data Engineer (Hyderabad)
🏢 Zorba AI
📍 Hyderabad