Description
Solution Architecture/Data Architecture
- Change Source code to accommodate for various architectural considerations e.g.: Moving MSAL based authentication to JWT based authentication to Databricks X-forwarded headers while using Apps
- Sit through Technical reviews for configuring necessary Azure resources for deploying the application.
- Prepare Technical/Network Architecture Documentation and send it reviews and approvals for cloud and security teams before going live.

Databricks
- Design and build pipelines for ingesting data into Databricks from various sources like SAP, website scraping. Core technical skill will be PySpark and PySQL
- Write and do scenario-based testing, edge cases and discuss with the stake holders to finalize the necessary code changes and acceptance criteria.
- Understanding of the Databricks Unity Catalog permissions model and how to use OBO tokens and configure Service Principals for Databricks Agents,Genie Spaces, Machine Learning and Foundation Models.
- Knowledge of Databricks Apps.



Ability to build and Deploy Databricks Apps using Visual Studio Code.
- Sending custom notifications from Databricks using APIs to custom Team Channel Webhooks

AI:
- Ability to use FastAPI /React front end and build minimal Single Page Applications(SPAs) using Chat UI interfaces and Claude Models/APIs
- Ability to use defined System Prompts, User Prompts etc..(i..e,) skills in prompt engineering.
- Basic Knowledge of Token Economics and ability use rule enforcement in prompts in addition to standard NLP language.
- Ability to reengineer code and see what it does and map it to the requirement specifications from users
- Ability to use Graphs to detect inter dependencies between rules and circular references.
- Ability to use both Claude Vision API in addition to standard text processing.
- Understand and develop Retrieval Augmented Generation specifically to Databricks like AI Search indexes, Hierarchical Chunking etc..
- Know how to implement

📌 Lead Data Engineer with AI (Bengaluru)
🏢 Enable Dataorporated
📍 Bengaluru