Role Overview

We are looking for a Databricks Engineer with strong experience in Azure data engineering, Databricks and data lake technologies , along with an understanding of IT Security/Cybersecurity use cases . The ideal candidate is a generalist who can work across security data engineering, analytics, Data Science and emerging GenAI use cases.

Immediate joiners preferred.

Key Responsibilities

- Design, develop and maintain scalable data pipelines for IT Security and Cybersecurity use cases .
- Build and manage data solutions using Azure Data Lake Storage (ADLS/ADLS Gen2), Azure Databricks and Azure data services .
- Ingest, transform and process security data from multiple sources such as SIEM, endpoint, identity, network, cloud and application security platforms .
- Develop Databricks/PySpark solutions for processing and analysing large volumes of security telemetry.
- Work with security logs, alerts, vulnerability data, threat intelligence, authentication data and other security datasets.
- Work with Azure Security services and understand security-related data generated across the Azure/Microsoft security ecosystem.
- Develop data models and curated datasets to support Security Operations, Threat Detection, Risk Analytics and reporting .
- Collaborate with Data Scientists to prepare datasets for machine learning, anomaly detection, behavioural analytics and predictive use cases .
- Support GenAI/LLM/RAG use cases involving security data, knowledge retrieval and intelligent security analytics.
- Develop automation scripts using Python
- Build and maintain Power BI dashboards and reports for security metrics, trends, risks and operational KPIs.
- Implement data quality, governance,



access controls and monitoring across data pipelines.
- Troubleshoot data ingestion, transformation and pipeline issues and optimize performance and cost.
- Collaborate with Cybersecurity, SOC, Data Engineering, Data Science and Cloud Engineering teams.

Required Skills & Experience

- 8+ years of experience in Data Engineering, Data Analytics, Security Data Engineering or a related field.
- Solid hands-on experience with Microsoft Azure data services.
- Strong experience with Azure Data Lake Storage (ADLS/ADLS Gen2) .
- Hands-on experience with Azure Databricks and PySpark .
- Strong SQL and Python skills.
- Experience working with IT Security/Cybersecurity data or security-related use cases .
- Good understanding of Azure Security / Microsoft security ecosystem .
- Experience in data ingestion, transformation, data modelling and pipeline development.
- Experience with Power BI dashboards and reporting .

Data Science / AI Exposure

- Good understanding of Data Science concepts , including statistics, machine learning, anomaly detection and feature engineering.
- Ability to work with Data Scientists and understand ML/data requirements.
- Exposure to GenAI, LLM, RAG or AI-powered analytics is preferred.
- Ability to explore and analyze large security datasets and identify meaningful patterns and trends.

Preferred Certifications:

- Databricks Certified Data Engineer Associate / Professional or equivalent Databricks certification.
- Microsoft security certifications such as SC-200 – Security Operations Analyst Associate , SC-500 – Cloud and AI Security Engineer Associate
- Other relevant certifications such as Security+, CEH, CISSP or equivalent are an advantage.

📌 Databricks Engineer (Bengaluru)
🏢 HireFlex
📍 Bengaluru