Role & responsibilities

- Design, develop, and maintain scalable data pipelines and data processing solutions.
- Build and optimize data solutions using Snowflake and dbt.
- Work extensively with AWS services, including S3, MWAA, MSK, and SageMaker.
- Develop large-scale data processing solutions using Hadoop, Hive, Spark, and PySpark.
- Build and manage data ingestion and streaming pipelines using Kafka/MSK.
- Develop and orchestrate workflows using Apache Airflow/AWS MWAA.
- Write and optimize complex queries using Advanced SQL.
- Develop data processing and automation solutions using Python and, where applicable, Scala.
- Work with modern data architectures and table/storage formats such as Apache Iceberg.
- Develop and maintain configuration and deployment components using YAML, Git, and Docker.




- Troubleshoot data pipeline issues and improve performance, scalability, and reliability.
- Collaborate with data, analytics, cloud, and business teams to deliver data engineering solutions.

Preferred candidate profile

- Strong hands-on experience with AWS
- Robust experience with Snowflake
- Experience with Hadoop/Big Data ecosystem
- Strong hands-on experience in Spark and/or PySpark
- Strong Python and SQL
- Experience developing and maintaining ETL/ELT data pipelines
- Good understanding of large-scale and distributed data processing

📌 Senior Data Engineer (Bengaluru)
🏢 Allegis Group
📍 Bengaluru