Vantara Metals runs six plants with OPC-UA historians, two ERP instances and a decade of spreadsheets. We are consolidating that into a lakehouse on Delta Lake, with about 420 dbt models feeding plant yield, energy cost per tonne and inventory ageing. The current nightly batch takes 3 hours 10 minutes and needs to become a 15-minute micro-batch without doubling the compute bill.
Full-time — on site
5-8 years building production data pipelines, including at least one platform you migrated while it was in use
Jodhpur, India
Apply for this role
The work
What you will do
Build ingestion from OPC-UA historians and ERP change-data-capture streams, handling clock skew and out-of-order events from sites with intermittent connectivity.
Restructure the dbt project into clearly layered staging, intermediate and mart models, with tests on every column a report depends on.
Cut the critical path from 3 hours 10 minutes to a 15-minute micro-batch through incremental models and partition pruning.
Define and enforce data contracts between the ingestion layer and the analytics team, including a deprecation process for breaking changes.
Own cost: track spend per pipeline and keep the monthly warehouse bill within an agreed ceiling.
You
What we look for
Solid SQL — window functions, incremental merge strategies, and knowing why your query scanned 400 GB when it should have scanned 4.
Spark or an equivalent distributed engine in production, including how you diagnosed a skewed join.
Python for orchestration and transformation, with tests that run in CI.
Dimensional modelling that survives contact with a client changing their reporting hierarchy mid-project.
Direct experience of a late-arriving or corrected record breaking a downstream figure, and what you built so it did not happen twice.
More roles
Other openings
Bengaluru, India
Full-time — hybrid, three days a week on site
Senior B
📌 Senior Data Engineer (Jodhpur)
🏢 Atmora Tech
📍 Jodhpur
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.