Data Engineer – Azure Databricks

Experience:
6+ Years

Location:
Bengaluru- Whitefield

Work Mode:
Hybrid – 3 days/week

Interview Process:
2 Technical Round

Job Summary

We are looking for an experienced
Data Engineer
with 6+ years of hands-on experience in building and managing scalable data engineering solutions. The ideal candidate should have robust expertise in
Spark, Scala, Airflow, Hive, Kubernetes, Iceberg, and Azure Databricks
.

The candidate should also have a good understanding of modern data platforms, cloud technologies, CI/CD practices, and the ability to leverage
AI across the Software Development Lifecycle (SDLC) and Data Engineering
to drive innovation and business value.

Key Responsibilities
- Design, develop, and maintain scalable and reliable data engineering pipelines.
- Build data processing solutions using
Apache Spark and Scala
.
- Develop and manage workflows using
Apache Airflow
.
- Work with
Hive, Iceberg, Kubernetes, and Docker
in modern data platforms.
- Develop and maintain CI/CD pipelines using
GitHub Actions
.




- Design and implement data solutions using
Azure Databricks
.
- Work with
ADLS, Azure Data Factory (ADF), Delta Lake, Databricks Jobs and Pipelines
.
- Utilize
Lakeflow Connect, Unity Catalog, and Databricks Compute (Classic & Serverless)
.
- Implement data governance, security, and access controls using Unity Catalog.
- Optimize data pipelines and processing workloads for performance and scalability.
- Collaborate with data scientists, architects, product teams, and other engineering teams.
- Apply AI capabilities and tools across the SDLC and Data Engineering lifecycle to improve productivity, automation, and business outcomes.
- Follow engineering best practices around version control, testing, deployment, monitoring, and documentation.

Required Skills

Must-Have – Priority 1
- Strong hands-on experience with
Apache Spark
- Scala
- Apache Airflow
- Hive
- Kubernetes
- Apache Iceberg
- Good knowledge of
Docker
- Good ex

📌 Senior Data Engineer (Bengaluru)
🏢 Quess IT Staffing
📍 Bengaluru