The Candidate Will Have Responsibilities Across The Following Functions
Databricks Solution Building and POC Development:
- Design and build Databricks-native solutions and POCs as part of the CoE practice, covering data ingestion, transformation, and AI- assisted analytics.
- Explore and validate current Databricks features -DLT, Lakeflow.
- Connect, Mosaic AI, Genie Spaces -and document findings with working demos or solution assets.
- Contribute reusable solution components and accelerators to the CoE solution catalogue.

Data Engineering
- Build and optimise data pipelines using PySpark, Delta Live Tables, and Databricks Workflows.
- Implement structured streaming pipelines for real-time data use cases.
- Apply Delta Lake best practices: partitioning, Z-ordering, vacuum, schema management.
- Work within Unity Catalogue to enforce data governance, lineage, and access control standards.

Mandatory Skills
The core requirements for the job include the following:
- Python and PySpark: Production-grade Python and PySpark -clean, modular, reusable code; not notebook-first scripting.




- SQL / SparkSQL: From fundamental queries through advanced joins, window functions, and complex transformations.
- Databricks Platform -depth across the core stack, not surface familiarity:
- Delta Lake: ACID transactions, schema evolution, time travel, optimisation.
- Unity Catalog -data governance, access control, lineage; understanding of how it structures a lakehouse.
- Delta Live Tables (DLT) -declarative pipeline development, expectations, data quality enforcement.
- Lakeflow Connect -hands-on exposure to ingestion connectors and pipeline orchestration within Databricks.
- Structured Streaming -real-time and near-real-time pipeline patterns on Databricks.
- Azure Data Factory: Pipeline orchestration, integration with Databricks; working knowledge of ADF triggers, linked services, and data movement patterns.
- ETL / ELT Fundamentals: Understands the difference between ETL and ELT.
- ETL a

📌 Azure Databricks Engineer (Bengaluru)
🏢 Netscribes
📍 Bengaluru