Data Engineer – Azure Databricks

Experience:

6+ Years Location:

Bengaluru- Whitefield Work Mode:

Hybrid – 3 days/week Interview Process:

2 Technical Round

Job Summary

We are looking for an experienced

Data Engineer

with 6+ years of hands-on experience in building and managing scalable data engineering solutions. The ideal candidate should have strong expertise in

Spark, Scala, Airflow, Hive, Kubernetes, Iceberg, and Azure Databricks . The candidate should also have a good understanding of modern data platforms, cloud technologies, CI/CD practices, and the ability to leverage

AI across the Software Development Lifecycle (SDLC) and Data Engineering

to drive innovation and business value.

Key Responsibilities Design, develop, and maintain scalable and reliable data engineering pipelines. Build data processing solutions using

Apache Spark and Scala . Develop and manage workflows using

Apache Airflow . Work with

Hive, Iceberg, Kubernetes, and Docker

in modern data platforms. Develop and maintain CI/CD pipelines using

GitHub Actions . Design and implement data solutions using

Azure Databricks . Work with

ADLS, Azure Data Factory (ADF), Delta Lake, Databricks Jobs and Pipelines . Utilize

Lakeflow Connect, Unity Catalog, and Databricks Compute (Classic & Serverless) . Implement data governance, security, and access controls using Unity Catalog. Optimize data pipelines and processing workloads for performance and scalability. Collaborate with data scientists, architects, product teams, and other engineering teams.



Apply AI capabilities and tools across the SDLC and Data Engineering lifecycle to improve productivity, automation, and business outcomes. Follow engineering best practices around version control, testing, deployment, monitoring, and documentation.

Required Skills Must-Have – Priority 1 Robust hands-on experience with

Apache Spark Scala Apache Airflow Hive Kubernetes Apache Iceberg Good knowledge of

Docker Good experience with

GitHub Actions / CI-CD Must-Have – Priority 2 Strong experience with

Azure Databricks ADLS Databricks Jobs & Pipelines Azure Data Factory (ADF) Delta Lake Lakeflow Connect Unity Catalog Databricks Compute –

Classic & Serverless Must-Have – Priority 3 Experience using

AI tools/capabilities within SDLC and Data Engineering Ability to identify opportunities to use AI for automation, development, optimization, and innovation Strong problem-solving and analytical skills

Additional Requirements 6+ years of relevant experience in Data Engineering. Strong understanding of data engineering concepts and distributed data processing. Excellent communication and collaboration skills. Candidates must be based in or willing to relocate to

Bengaluru . Candidates should be comfortable working from the office Willingness to attend

2 technical interview rounds .

Preferred Candidate Profile

Candidates with strong hands-on experience across

Spark/Scala/Airflow

and the

Azure Databricks ecosystem

will be given priority. Experience in modern data lakehouse technologies, containerization, CI/CD, and practical application of AI in engineering environments will be an added advantage.

📌 Senior Data Engineer (Mumbai)
🏢 Quess IT Staffing
📍 Mumbai