Job Information
- Work Experience 7 - 9 Years
- Date Opened 08/11/2026
- Industry IT Services
- Job Type Full Time Employment
- Remote Job
Job Description
This is a remote position.
Data Engineering & Platform
- Medallion pipeline ownership —Build, maintain, and optimize Bronze, Silver, and Gold layer pipelines in Microsoft Fabric using ADF (Azure Data Factory) and PySpark notebooks. Own the full ingestion-to-consumption chain for all source systems including Acumatica ERP, HubSpot CRM, AutoStore, RingCentral, website analytics, and marketplace channel data (Net32, Amazon, eBay, and others).
- Event-driven ingestion —Implement and maintain event-triggered ADF pipelines using Acumatica SaaS push notifications routed through Azure Service Bus and Azure Function App receivers. Build OData watermark fallback pipelines with LastModifiedDateTime-based incremental load, pagination handling, and session concurrency management.
- CDC and incremental load —Design and manage Change Data Capture strategies including CDF (Change Data Feed) versioning in the watermark registry,
last-processed version tracking, and initial vs. incremental load logic. Ensure Silver batching strategy (currently 37 tables across 3 batches) runs within SLA and scale intelligently as current sources are onboarded.
- Bronze-to-Gold transformation —Develop and maintain Silver transformation notebooks including deduplication, null handling, type casting, and DQ rule application. Build and optimize Gold KPI layer notebooks following a star schema — fact and dimension tables — with orchestration notebooks managing execution sequence and dependency management.
- Schema management and evolution —Validate Bronze Delta table schemas against source system payloads. Implement schema evolution handling via the Delta transaction log. Maintain the source_object_config and watermark_registry control tables that govern what is ingested and how.
- Data quality enforcement —Implement and maintain DQ rules (DQ-001 throu
📌 Senior Data Engineer (India)
🏢 Veltris
📍 India