Role & Responsibilities;

We are looking for an experienced Azure Databricks Data Engineer with 6-8 years of experience in designing, developing, and supporting scalable data engineering solutions.

The candidate will be responsible for:

- Designing and developing scalable data pipelines using Azure Databricks, PySpark, and Azure Data Factory.
- Building ADF pipelines for ingestion, orchestration, scheduling, dependency management, and monitoring.
- Developing complex PySpark transformations for batch and large-volume data processing.
- Implementing Delta Lake and Lakehouse architecture using Bronze, Silver, and Gold layers.
- Designing ETL and ELT workflows for structured and semi-structured data.
- Writing and optimizing complex SQL queries.
- Improving Spark and PySpark performance through partitioning, caching, join optimization, cluster tuning, and efficient file handling.
- Implementing Delta Lake features such as schema evolution, MERGE, time travel, OPTIMIZE, VACUUM, and change data capture.
- Managing data access, governance, lineage, and security using Unity Catalog.
- Developing reusable frameworks for ingestion, transformation, auditing, logging, error handling, and data-quality validation.
- Implementing CI/CD processes for Databricks using Azure DevOps and Git.
- Troubleshooting production issues, pipeline failures, performance bottlenecks, and data-quality problems.
- Supporting deployment, monitoring, incident resolution, and root-cause analysis.




- Collaborating with architects, analysts, business stakeholders, and cross-functional teams.
- Preparing technical documentation, design documents, runbooks, and deployment guides.

Preferred Candidate Profile
- 68 years of overall experience in data engineering.
- Strong hands-on experience in Azure Databricks and PySpark development.
- Strong experience in Azure Data Factory pipeline development and orchestration.
- Good understanding of Databricks architecture, Spark architecture, clusters, jobs, notebooks, and workflows.
- Hands-on experience with Delta Lake, Lakehouse architecture, and Medallion architecture.
- Practical experience with Unity Catalog, data governance, role-based access control, security, and data lineage.
- Solid SQL development and query-optimization skills.
- Experience in Spark and PySpark performance tuning.
- Experience designing data models and end-to-end ETL/ELT frameworks.
- Hands-on experience implementing CI/CD using Azure DevOps.
- Strong problem-solving, debugging, and production-support skills.
- Good communication and stakeholder-management abilities.
- Candidates must be available to join immediately.
- Candidates must be genuinely interested and fully committed to the opportunity.
- Candidates should be comfortable working from any approved BSL location.
- Candidates should have a stable employment history and a low risk of offer withdrawal.

📌 Azure Databricks Data Engineer (Bengaluru)
🏢 Bhuvitech Innovations
📍 Bengaluru