Role & responsibilities

- Design, develop, and maintain scalable data pipelines and data processing solutions.

- Build and optimize data solutions using Snowflake and dbt.

- Work extensively with AWS services, including S3, MWAA, MSK, and SageMaker.

- Develop large-scale data processing solutions using Hadoop, Hive, Spark, and PySpark.

- Build and manage data ingestion and streaming pipelines using Kafka/MSK.

- Develop and orchestrate workflows using Apache Airflow/AWS MWAA.

- Write and optimize complex queries using Advanced SQL.

- Develop data processing and automation solutions using Python and, where applicable, Scala.

- Work with modern data architectures and table/storage formats such as Apache Iceberg.

- Develop and maintain configuration and deployment components using YAML, Git, and Docker.





- Troubleshoot data pipeline issues and improve performance, scalability, and reliability.

- Collaborate with data, analytics, cloud, and business teams to deliver data engineering solutions.

Preferred candidate profile

- Strong hands-on experience with AWS

- Strong experience with Snowflake

- Experience with Hadoop/Big Data ecosystem

- Strong hands-on experience in Spark and/or PySpark

- Strong Python and SQL

- Experience developing and maintaining ETL/ELT data pipelines

- Positive understanding of large-scale and distributed data processing

📌 Senior Data Engineer (Bengaluru)
🏢 Allegis Group
📍 Bengaluru