What success looks like in this role:
End-to-End Pipeline Engineering: Build and automate robust ETL/ELT pipelines using Azure Data Factory (ADF), AWS Glue, and Apache Airflow.
- Distributed Computing: Develop large-scale data processing jobs using PySpark and Scala within Databricks or EMR environments.
- Streaming & Real-time Integration: Design and implement real-time data ingestion and processing layers using Apache Kafka, Confluent, or AWS Kinesis.
- Data Lakehouse : Manage and optimize cloud storage using ADLS Gen2 and S3, implementing ACID transactions with Delta Lake or Apache Iceberg.
- Advanced Data Modeling: Design highly performant schemas for cloud data warehouses like Snowflake, Amazon Redshift, or Google BigQuery.
- Data Transformation & Quality: Use dbt (data build tool) for modeling and implement automated quality checks using Great Expectations or Soda.
- Infrastructure & CI/CD: Deploy and manage data infrastructure using Terraform or CloudFormation, and maintain CI/CD pipelines via GitHub Actions or GitLab CI.

Technical Stack Requirements
- Cloud Platforms:



Deep hands-on experience with Microsoft Azure (ADF, Synapse, Databricks) and AWS (S3, Glue, Athena, Lambda).
- Programming: Strong proficiency in Python (PySpark, FastAPI), SQL, and familiarity with Java or Scala.
- Big Data Tools: Experience with Apache Spark, Apache Flink, and Hadoop ecosystem.
- Databases: Robust knowledge of both Relational (PostgreSQL, MySQL) and NoSQL (MongoDB, Cassandra, or DynamoDB) databases.
- Containerization: Proficiency with Docker and Kubernetes (K8s) for deploying data services.
- Observability: Familiarity with monitoring tools like Prometheus, Grafana, or Datadog to track pipeline health.

LI-SS1

You will be successful in this role if you have:
Experience: 2-4years of professional experience in data engineering, backend engineering, or a related field.
- Education: Bachelor’s Engineering,
- Methodology: Strong understanding of Agile methodologies and the abil

📌 Azure Data Engineer (Bengaluru)
🏢 Unisys
📍 Bengaluru