Baazi Games is looking for a Data Engineer II – AWS Specialist with strong hands-on experience in building and scaling batch and real-time data pipelines .
The ideal candidate should have strong expertise in Python, Apache Spark, Kafka, Airflow, Apache Iceberg, and AWS analytics services , with experience working on scalable and production-grade data systems.
Key Responsibilities
- Build and scale batch and real-time data pipelines using Python and Apache Spark on AWS EMR .
- Design and maintain high-throughput streaming pipelines using Apache Kafka .
- Maintain and optimise transactional data lake storage using Apache Iceberg .
- Design, schedule, and monitor complex workflows using Apache Airflow .
- Build event-driven data automation using AWS Lambda, EventBridge, and SNS .
- Catalog and manage datasets using AWS Glue .
- Work with Amazon S3 for scalable and optimised data storage.
- Manage secure cloud infrastructure and access using AWS IAM .
- Troubleshoot and optimise data pipelines for reliability, scalability, and performance.
Required Skills
- 3–5 years of professional Data Engineering experience.
- Strong programming skills in Python .
- Hands-on experience with Apache Spark / PySpark .
- Robust experience with Apache Kafka and real-time data processing.
- Experience with Apache Airflow for workflow orchestration.
- Hands-on experience with Apache Iceberg and modern data lake/lakehouse architecture.
- Strong knowledge of AWS data and analytics services .
- Experience with AWS EMR, S3, Glue, Lambda, EventBridge, SNS, and IAM .
- Good understanding of batch and streaming architectures .
- Strong problem-solving and debugging skills.