KEY RESPONSIBILITIES
- Design & implement governed ingestion pipelines consuming in office & off-site events into canonical

event schema
- Build and maintain Kafka or Kinesis-based streaming pipelines with schema validation, data quality

checks, and alerting on source failures
- Integrate batch ingestion from databases and alongside the streaming path, managing orchestration

via Airflow or equivalent
- Enforce privacy flags at ingestion time quarantining or anonymising events without valid consent

before they reach downstream layers
- Maintain a raw event store with partitions by date and source, serving as the audit source for the

pipeline
- Implement deterministic identity across fragmented systems
- Build and maintain the cookie-to-login stitching mechanism
- Design identity logic spanning on-site sessions, CRM records etc
- Monitor identity resolution quality match rate, false positive rate, unresolved session ratio — and

iterate on matching logic to improve coverage over time
- Compute windowed aggregations over the event store
- Build and maintain governed,



versioned feature definitions in a feature store ensuring normalisation,

encoding, and embedding lookup logic is consistent between training and serving pipelines
- Collaborate with data scientists to translate model feature requirements into production-grade

pipeline implementations with no training-serving skew
- Implement automated data contract validation between pipeline layers, schema compatibility checks,

completeness assertions, and anomaly detection
- Define and enforce SLAs on pipeline freshness, completeness, and accuracy with monitoring

dashboards and escalation paths for breaches
- Ensure PII is classified, tagged, and handled according to GDPR and CCPA requirements at every stage

of the pipeline like ingestion, storage, feature compute, and serving
- Contribute and maintain data lineage document, full traceability from raw event to feature to model

prediction
- Work closely with the

📌 Senior/Lead Azure Data Engineer (Bengaluru)
🏢 Creative Synergies
📍 Bengaluru