Senior Data Engineer to build the pipelines that turn raw multimodal capture — audio, egocentric video and sensor streams from 20+ countries — into versioned, quality-checked, client-ready datasets for frontier AI and robotics teams.

Location: Bengaluru, Karnataka, India · Employment: full time, on-site · Experience: 5+ years

WHAT YOU WILL DO

Design and build batch and streaming pipelines that ingest raw audio, video and sensor (IMU) data from the global capture network into cloud storage.

- Build processing stages for unstructured media: transcoding, segmentation, metadata extraction, deduplication and joins with annotation and contributor data.
- Own data quality: automated validation, QC gates and anomaly detection so that bad captures are caught before they reach a client.
- Build dataset versioning,



lineage and manifests so any delivered dataset can be reproduced exactly.
- Package and deliver datasets in the formats client teams ask for (Parquet, WebDataset, Hugging Face datasets, MCAP and similar), on time and to specification.

WHAT YOU NEED

5+ years of data engineering, building production pipelines end to end

Strong Python and SQ

- Distributed processing at scale — Spark, Ray, Beam or Dask
- Workflow orchestration (Airflow, Dagster or Prefect) and cloud object storage on AWS or GCP
- Has worked with large unstructured or media data (audio, video, images, sensor logs), not only tabular warehouse data

📌 Senior Data Engineer (Bengaluru)
🏢 Excelon Solutions
📍 Bengaluru