Data Engineer – Azure Databricks

Experience:
6+ Years

Location:
Bengaluru- Whitefield

Work Mode:
Hybrid – 3 days/week

Interview Process:
2 Technical Round

Job Summary

We are looking for an experienced
Data Engineer
with 6+ years of hands-on experience in building and managing scalable data engineering solutions. The ideal candidate should have robust expertise in
Spark, Scala, Airflow, Hive, Kubernetes, Iceberg, and Azure Databricks
.

The candidate should also have a good understanding of up-to-date data platforms, cloud technologies, CI/CD practices, and the ability to leverage
AI across the Software Development Lifecycle (SDLC) and Data Engineering
to drive innovation and business value.

Key Responsibilities
Design, develop, and maintain scalable and reliable data engineering pipelines.
Build data processing solutions using
Apache Spark and Scala
.
Develop and manage workflows using
Apache Airflow
.
Work with
Hive, Iceberg, Kubernetes, and Docker
in modern data platforms.
Develop and maintain CI/CD pipelines using
GitHub Actions
.




Design and implement data solutions using
Azure Databricks
.
Work with
ADLS, Azure Data Factory (ADF), Delta Lake, Databricks Jobs and Pipelines
.
Utilize
Lakeflow Connect, Unity Catalog, and Databricks Compute (Classic & Serverless)
.
Implement data governance, security, and access controls using Unity Catalog.
Optimize data pipelines and processing workloads for performance and scalability.
Collaborate with data scientists, architects, product teams, and other engineering teams.
Apply AI capabilities and tools across the SDLC and Data Engineering lifecycle to improve productivity, automation, and business outcomes.
Follow engineering best practices around version control, testing, deployment, monitoring, and documentation.

Required Skills

Must-Have – Priority 1
Strong hands-on experience with
Apache Spark
Scala
Apache Airflow
Hive
Kubernetes
Apache Iceberg
Good knowledge of
Docker
Valuable ex

📌 Senior Data Engineer Bengaluru
🏢 Quess IT Staffing
📍 Bengaluru