Role: Senior Azure Databricks Developer
Role Summary:
Senior Databricks Engineer with solid expertise in developing, optimizing, and maintaining data engineering solutions on the Databricks platform. The candidate should have deep experience with Apache Spark (PySpark), Databricks notebooks, Delta Lake, and Spark performance optimization. This role is focused on Spark-native data engineering rather than ETL tool development.
Must Have Skills:
Solid hands-on experience with Databricks notebooks, Jobs, Workflows, Repos, and Unity Catalog
Robust expertise in PySpark and Spark SQL
Experience building scalable Spark-based data pipelines
Strong understanding of Spark DataFrames, joins, aggregations, window functions, and distributed processing
Experience with Delta Lake (MERGE, OPTIMIZE, ZORDER, VACUUM, Change Data Feed)
Hands-on experience in Spark job performance tuning
Experience optimizing partitions, shuffles, caching, memory usage, and cluster utilization
Experience developing batch and incremental data processing pipelines
Strong programming skills in Python,



PySpark, and SQL
Experience working on Azure Databricks, AWS Databricks, or GCP Databricks
Experience with Git, CI/CD, code reviews, testing, and production deployments
Implement event-driven data flows and real-time data ingestion using Apache Kafka.
Develop and integrate REST APIs for both data extraction and downstream data serving.
Monitor, tune, and optimize PySpark workloads for maximum performance and cost efficiency.
Communicate effectively with cross-functional stakeholders to gather requirements and translate them into scalable technical solutions.
Mentor junior developers, guide architectural decisions, and prepare to take on team lead responsibilities in the future. as nice to have.

Positive to Have Skills:
Experience with Lakehouse and Medallion architecture
Knowledge of Data Modeling and Data Governance

Nice to Have Skills:
Experience with Delta Live Tables (

📌 Senior Azure Databricks Developer Pune
🏢 NewVison
📍 Pune