Key Responsibilities
- Design, develop and maintain scalable data engineering solutions using Apache Spark and Scala.
- Build and manage data pipelines using Airflow and related orchestration technologies.
- Work extensively with Hive, Kubernetes, Docker and GitHub Actions.
- Develop and optimize data solutions using Apache Iceberg.
- Design and implement data engineering solutions on Azure Databricks.
- Work with ADLS, ADF, Delta Lake, Databricks Jobs & Pipelines, Lakeflow Connect and Unity Catalog.
- Work with both Classic and Serverless Databricks Compute environments.
- Implement scalable, reliable and high-performance data pipelines.
- Follow CI/CD and DevOps best practices using GitHub Actions.
- Explore and implement AI-assisted development practices across SDLC and data engineering.
- Collaborate with cross-functional teams to deliver innovative data solutions aligned with business requirements.

Must-Have Skills

1. Spark, Scala, Airflow, Hive
2.



Azure Databricks, ADLS, ADF, Delta Lake
3. Kubernetes, Docker, GitHub Actions

Strongly Preferred Skills
- Apache Iceberg
- Databricks Jobs & Pipelines
- Lakeflow Connect
- Unity Catalog
- Databricks Classic & Serverless Compute
- Azure Data Engineering
- CI/CD
- AI-assisted SDLC / AI for Data Engineering

Candidate Profile
- 6+ years of overall experience in Data Engineering.
- Strong hands-on experience with Spark and Scala.
- Valuable experience in Azure Databricks ecosystem.
- Strong understanding of data pipelines, orchestration and distributed data processing.
- Experience with containerization and DevOps tools.
- Ability to work from Whitefield office, Bangalore, 3 days per week.
- Candidates should be available for the Saturday interview.

📌 Senior Data Engineer Spark Scala Azure Databricks (Bengaluru)
🏢 Quess
📍 Bengaluru