Job Description

Nielsen’s Digital Content AI team builds and operates data-processing pipelines that turn digital audience and interaction signals into reliable, analytics-ready datasets. As a principal-level individual contributor, you will bring a Solution first mindset and strong data understanding to the full lifecycle: understand new data sources, build reusable ingestion and transformation workflows, validate and enrich data, publish it to downstream data platforms, and keep production processing dependable. You will lead multiple cross-functional projects through measurable business deliverables, providing technical direction through expertise and influence rather than formal people management. You will work across mobile, Connected TV (CTV), desktop/laptop and web use cases, partnering with product, data collection, platform, quality and downstream analytics teams.

● Lead the technical design and delivery of batch and near-daily data pipelines, from raw inputs through curated outputs and publication to the Master Data Layer (MDL).
● Profile and interpret incoming telemetry from diverse data sources; identify data gaps and translate source formats into dependable processing logic.
● Apply an AI-first approach to engineering and operations: identify appropriate opportunities for AI-assisted data exploration, code and test generation, classification, anomaly detection and workflow automation; validate outputs rigorously and use AI responsibly with appropriate privacy and security controls.
● Design and operate real-time and streaming event workflows alongside batch pipelines, managing event validation, schema evolution, ordering and duplication,



late or replayed events, processing latency, backpressure and failure recovery.
● Build and maintain parsers, configuration-driven workflows, event/session logic, business rules and schema mappings for application usage and in-app activity across mobile, CTV, desktop/laptop and web products.
● Develop data-stitching and transformation logic that correlates relevant sources, derives sessions and engagement measures, and produces consistent intermediate and curated datasets.
● Deliver data processing for streaming/video, e-commerce and advertising use cases, including event classification, purchase and product/category/search activity, app and web usage, and incremental dataset updates.
● Integrate reference and enrichment data—including content metadata and classification inputs—where required; collaborate with machine-learning and platform teams to operationalize model-based classification and enrichment.
● Build reusable frameworks for onboarding new applications, platforms, data sources and markets; support configuration, historical backfills, reprocessing and safe daily refreshes.
● Orchestrate and operate workflows using tools such as Airflow and AWS workflow services; monitor schedules, dependencies, retries, failures and data availability,



and troubleshoot production issues through root-cause analysis.
● Implement and evolve automated data-quality controls, including schema and completeness checks, freshness and volume monitoring, null/range/duplicate checks, event consistency, stitching validation and reconciliation between processing layers and MDL.
● Create and use operational dashboards, alerts and job-level telemetry to make pipeline health, data quality, performance and processing costs visible; improve reliability and efficiency based on those signals.
● Optimize distributed processing and cloud data layouts for correctness, scalability, runtime and cost, working with technologies such as Spark, Parquet, Amazon S3, AWS Glue, Athena and EMR.
● Contribute to design reviews, code reviews, testing strategy, release readiness, runbooks and technical documentation; maintain explicit data contracts, schemas and operational ownership boundaries.
● Provide technical direction and mentoring to engineers, lead moderately complex initiatives through influence, and communicate risks, dependencies, estimates and delivery status to cross-functional partners.
● Drive multiple projects or workstreams end to end, translating business needs into scope, milestones and technical plans; coordinate dependencies and stakeholders; and remain accountable for quality, adoption, timelines and measurable business outcomes—not people-management responsibilities.
● Handle sensitive data responsibly and follow Nielsen’s security, privacy, access-control and engineering practices.

📌 Principal Member Technical Staff - Data engineer (Bengaluru)
🏢 The Nielsen
📍 Bengaluru