Broad Function:

The Senior Data Engineer – VARTA SENSE will be responsible for designing, developing, and managing scalable and reliable data pipelines that form the core data foundation of the VARTA SENSE platform.

The role will work closely with the ML Lead, Solution Architect, Backend Engineering team and other technology stakeholders to transform complex banking and transaction data into standardized, reusable and production-ready datasets and features.

The position requires robust hands-on expertise in Python, SQL, Spark/PySpark, data pipelines, orchestration, data quality and distributed data processing, with particular emphasis on reliability, performance, scalability and secure deployment within restricted banking settings.

Roles and Responsibilities (not limited to):
Data Pipeline Development & Engineering
Design and develop scalable batch, incremental and production-grade data pipelines for large volumes of transaction and customer data.
Build standardized and reusable data models and canonical schemas for transactions, customer attributes, offers,



exposure and outcome events.
Develop configurable ingestion and mapping frameworks to integrate data received from different banks and source systems.
Manage pipeline orchestration including dependencies, checkpointing, retries, partial failures, backfills and protected replay mechanisms.
Implement deduplication, late-arriving data handling, incremental loads, CDC, watermarking and merge/upsert processing.
Data Quality, Reconciliation & Governance
Establish strong data-quality controls, automated testing, source-to-target reconciliation and schema validation.
Ensure data lineage and schema evolution are properly managed across pipelines.
Proactively identify and prevent issues such as missing data, duplicate records, incorrect aggregations, silent data loss and double counting.
Maintain reliable and auditable data-processing standards across production environments.
ML Feature En

📌 Senior Data Engineer Noida (India)
🏢 Friends Color Images
📍 India