Data Platform Engineer

Location-Remote

Role Summary

We are seeking Senior Data Platform Engineers to join our CSO team. This is a dual-domain environment spanning two fully segregated platforms built on Azure Databricks and the Medallion Architecture: the CSO Security Data Platform and the SOX Application Data Platform (fully segregated to satisfy Sarbanes-Oxley requirements). Engineers are the backbone of both platforms — ensuring data quality, enabling security consumers, onboarding new sources, maintaining pipelines, and supporting SOX audits.

Key Responsibilities

Responsibilities below apply across the team. Individual engineers own the areas aligned to their role above.

1. Security Platform — Pipeline Operations & Data Quality

Monitor end-to-end Medallion pipeline health (L0→L3) across data sources, maintaining 99.5%+ success rate; triage quarantine spikes, dead-letter accumulation, and schema drift.

Schedule and manage Delta Lake maintenance (OPTIMIZE, VACUUM, Z-ORDER), enforce ADLS lifecycle policies,



and track DBU consumption and monthly storage costs.

Maintain Source Health Dashboard and Audit Control Tables as the single-pane view of all source delivery status, quality metrics, and pipeline state.

2. Security Platform — Customer Enablement

Act as primary support for SOC Analysts, IR/DFIR Investigators, SOC Leadership, Detection Engineers, and Compliance Teams on data discovery, ad hoc queries, and Genie Spaces.

Build and maintain AI/BI Dashboards and Power BI reports; curate Genie Space instructions, sample questions, and verified SQL per persona.

Support data discovery via MS Purview, triage enrichment gaps, and conduct training and onboarding for new platform consumers.

3. Security Platform — New Source Onboarding

Lead end-to-end onboarding of current security log sources: define schema profile, dedup key, partition key, OCSF mapping, quarantine strategy, and pipeline config per the Source Onboarding Framework

📌 Data Platform Engineer (India)
🏢 Cyber1Armor
📍 India