Baazi Games is looking for a Data Engineer II – AWS Specialist with strong hands-on experience in building and scaling batch and real-time data pipelines.
The ideal candidate should have strong expertise in Python, Apache Spark, Kafka, Airflow, Apache Iceberg, and AWS analytics services, with experience working on scalable and production-grade data systems.
Key Responsibilities
- Build and scale batch and real-time data pipelines using Python and Apache Spark on AWS EMR.
- Design and maintain high-throughput streaming pipelines using Apache Kafka.
- Maintain and optimise transactional data lake storage using Apache Iceberg.
- Design, schedule, and monitor complex workflows using Apache Airflow.
- Build event-driven data automation using AWS Lambda, EventBridge, and SNS.
- Catalog and manage datasets using AWS Glue.
- Work with Amazon S3 for scalable and optimised data storage.
- Manage secure cloud infrastructure and access using AWS IAM.
- Troubleshoot and optimise data pipelines for reliability, scalability, and performance.
Required Skills
- 3–5 years of qualified Data Engineering experience.
- Strong programming skills in Python.
- Hands-on experience with Apache Spark / PySpark.
- Strong experience with Apache Kafka and real-time data processing.
- Experience with Apache Airflow for workflow orchestration.
- Hands-on experience with Apache Iceberg and modern data lake/lakehouse architecture.
- Strong knowledge of AWS data and analytics services.
- Experience with AWS EMR, S3, Glue, Lambda, EventBridge, SNS, and IAM.
- Good understanding of batch and streaming architectures.
- Strong problem-solving and debugging skills.