23 Oct
|
IMR Soft
|
Bengaluru
23 Oct
IMR Soft
Bengaluru
Senior Data Engineer – AdTech Experience: 8+ years
Hybrid
Bangalore (Must have experience in the AdTech or Fintech domain)
We're looking for a Senior Data Engineer to design, build, and operate the data pipelines and aggregates that power AdTech platform. This is a hands-on individual contributor role focused on high-scale batch + streaming data processing, reliable data products, and analytics-ready datasets that enable prediction, agentic workflows, BI reporting, and measurement. You will partner closely with backend, ML, and product teams to deliver trusted, well-modeled data with strong performance, quality, and observability.
Build data pipelines: Develop robust batch and streaming pipelines (Kafka/Kinesis) to ingest, transform, and enrich large-scale event data (impressions, clicks, conversions, costs, identity signals).
- Create data aggregates & marts: Data modeling & contracts: Define schemas, data contracts, and versioning strategies to keep downstream systems stable as sources evolve.
- Data quality & reliability: Implement validation, anomaly detection, backfills, and reconciliation to ensure completeness, correctness, and timeliness (SLAs/SLOs).
- Performance & cost optimization: Airflow, Dagster, Step Functions) and CI/CD for data pipelines.
- Observability for data systems: Instrument pipelines with metrics, logs, lineage, and alerting to accelerate detection and root-cause analysis of data issues.
- 5+ years building production data pipelines and data products (batch and/or streaming)
in a high-scale environment.
- Solid experience with SQL and data modeling (dimensional modeling, star/snowflake schemas, event modeling).
- Hands-on experience with streaming systems (Kafka preferred) and/or AWS Kinesis, including event-driven designs.
- Proficiency in one or more languages used for data engineering (Python, Java, Scala, or Go).
- Experience with distributed data processing (Spark, Flink, or equivalent) and performance tuning at scale.
- Experience with AWS data services and cloud-native patterns (S3, Glue/EMR, Athena, Redshift, etc. Experience with orchestration/workflow tooling (Airflow/Dagster/Step Functions) and CI/CD for data workloads.
- Strong data quality/observability practices (tests, monitoring, lineage; Experience with SQL + NoSQL data stores (e.g., Postgres/MySQL; able to work with mixed audiences and translate needs into reliable data interfaces.
- DSP/SSP/exchange/RTB concepts and data flows.
- Experience building measurement pipelines (attribution, incrementality, lift, or experimentation analytics).
- Experience supporting ML feature stores, offline/online feature generation, or model training datasets.
- Experience with real-time analytics stores (Druid/ClickHouse/Pinot) and high-cardinality aggregation strategies.
- Deep knowledge of data governance/privacy, including PII handling and consent-aware data processing.
- BS/MS in CS/Engineering or equivalent practical experience.
📌 Senior Data Engineer - Hybrid (Bengaluru)
🏢 IMR Soft
📍 Bengaluru