Senior Data Engineer to build the pipelines that turn raw multimodal capture — audio, egocentric video and sensor streams from 20+ countries — into versioned, quality-checked, client-ready datasets for frontier AI and robotics teams.

Location:
Bengaluru, Karnataka, India
· Employment: full-time, on-site
·

Experience:
5+ years WHAT YOU WILL DO Design and build batch and streaming pipelines that ingest raw audio, video and sensor (IMU) data from the global capture network into cloud storage.
• Build processing stages for unstructured media: transcoding, segmentation, metadata extraction, deduplication and joins with annotation and contributor data.
• Own data quality: automated validation, QC gates and anomaly detection so that bad captures are caught before they reach a client.
• Build dataset versioning,



lineage and manifests so any delivered dataset can be reproduced exactly.
• Package and deliver datasets in the formats client teams ask for (Parquet, WebDataset, Hugging Face datasets, MCAP and similar), on time and to specification. WHAT YOU NEED 5+ years of data engineering, building production pipelines end to end Robust Python and SQ
• Distributed processing at scale — Spark, Ray, Beam or Dask
• Workflow orchestration (Airflow, Dagster or Prefect) and cloud object storage on AWS or GCP
• Has worked with large unstructured or media data (audio, video, images, sensor logs), not only tabular warehouse data

📌 Senior Data Engineer (Bengaluru)
🏢 Excelon Solutions
📍 Bengaluru