Job DescriptionData Engineer – Azure Databricks

NExperience: 6+ Years

NLocation: Bengaluru- Whitefield

NWork Mode: Hybrid – 3 days/week

NInterview Process: 2 Technical Round

NJob Summary

nWe are looking for an experienced Data Engineer with 6+ years of hands-on experience in building and managing scalable data engineering solutions. The ideal candidate should have strong expertise in Spark, Scala, Airflow, Hive, Kubernetes, Iceberg, and Azure Databricks.

NThe candidate should alsohave a good understanding of modern data platforms, cloud technologies, CI/CD practices, and the ability to leverage AI across the Software Development Lifecycle (SDLC) and Data Engineering to drive innovation and business value.

NKey Responsibilities

n

N
- Design, develop, and maintain scalable and reliable data engineering pipelines.
N
- Build data processing solutions using Apache Spark and Scala.
N
- Develop and manage workflows using Apache Airflow.
N
- Work with Hive, Iceberg, Kubernetes, and Docker in modern data platforms.
N
- Develop and maintain CI/CD pipelines using GitHub Actions.
N
- Design and implement data solutions using Azure Databricks.
N
- Work with ADLS, Azure Data Factory (ADF), Delta Lake, Databricks Jobs and Pipelines.
N
- Utilize Lakeflow Connect, Unity Catalog, and Databricks Compute (Classic & Serverless).
N
- Implement data governance, security, and access controls using Unity Catalog.
N
- Optimize data pipelinesand processing workloads for performance and scalability.
N
- Collaborate with data scientists, architects, product teams, and other engineering teams.
N
- Apply AI capabilities and tools across the SDLC and Data Engineering lifecycle to improve productivity,



automation, and business outcomes.
N
- Follow engineering bestpractices around version control, testing, deployment, monitoring, and documentation.
N

nRequired Skills

NMust-Have – Priority 1

N

n
- Strong hands-on experience with Apache Spark
N
- Scala
n
- ApacheAirflow
N
- Hive
n
- Kubernetes
N
- Apache Iceberg
n
- Positive knowledge of Docker
N
- Good experience with GitHub Actions / CI-CD
N

nMust-Have – Priority 2

N

n
- Strong experiencewith Azure Databricks
N
- ADLS
n
- Databricks Jobs & Pipelines
N
- Azure Data Factory (ADF)
N
- Delta Lake
n
- Lakeflow Connect
N
- Unity Catalog
n
- Databricks Compute – Classic & Serverless
N

nMust-Have – Priority 3

N

n
- Experience using AI tools/capabilities within SDLC and Data Engineering
N
- Ability to identify opportunities to use AI for automation, development, optimization, and innovation
N
- Strong problem-solving and analytical skills
N

nAdditional Requirements

N

n
- 6+ years of relevant experience in Data Engineering.
N
- Strong understanding ofdata engineering concepts and distributed data processing.
N
- Excellent communicationand collaboration skills.
N
- Candidates must be based in or willing to relocate to Bengaluru.
N
- Candidates should be comfortable working from the office
N
- Willingness to attend 2technical interview rounds.
N

nPreferred Candidate Profile

NCandidates with strong hands-on experience across Spark/Scala/Airflow and the Azure Databricks ecosystem will be given priority. Experience in modern data lakehouse technologies, containerization, CI/CD, and practical application of AI in engineering environments will be an added advantage.

📌 Hiring: Senior Data Engineer (Bengaluru)
🏢 Quess IT Staffing
📍 Bengaluru