23 Oct
|
Seventh Contact Hiring Solutions
|
Pune
23 Oct
Seventh Contact Hiring Solutions
Pune
Looking for A data engineer who models data as carefully as they run infrastructure, has dealt with revisions and point-in-time correctness in real pipelines, and has operated a graph database in production.
Responsibilities
● Data landing: connector framework with data contracts, delivery ledger, validation and quarantine; connectors for licensed vendor files and open sources; every record stamped with its three clocks (period, knowledge time, question time).
● Point-in-time store on object storage (Delta Lake): append-only revisions and vintages, an as-of read path that cannot return later-published information, staleness caps, lineage.
● Reference and identity layer: airports, lanes, corridors, regions, index hierarchy; identity resolution across sources with validity windows (Splink or rule-based).
● Graph layer (owner): generate the property-graph schema from the ontology (schema-as-code, LinkML); deterministic projections per as-of date; load and operate Neo4j (Aura) with a read-only product projection; write graph features back into the feature store as as-of columns; enforce that no licensed raw value or forecast leaks into the graph; keep the rebuild to one command and the GPU/graph-ML path open.
● Platform:
Terraform on Google Cloud (or Azure/AWS), container jobs, Postgres with row-level security, orchestration with as-of partitions (Dagster or comparable), MLflow, CI/CD gates, observability, secrets, backups, cost control; keep the repository agent-ready (dev container, fixtures, contract tests, leak scans).
Must have
● Solid Python and SQL; DuckDB/Polars/Pandas; larger data pipelines with data contracts, schema evolution and data-quality checks.
● Point-in-time data correctness: as-of reads, publication lags, vintages and revisions, reproducibility.
● Production experience with Neo4j and Cypher: property-graph data modelling, loading at scale, query performance, operating a managed instance; understands the graph as a projection of tabular truth, not a system of record.
● Docker, GitHub Actions, Terraform, one major cloud (GCP preferred), PostgreSQL, Parquet/Delta Lake, Dagster/Airflow, MLflow, logging and monitoring.
● Can turn "build a new macro-data connector" into a controlled agent workflow: spec → connector → fixtures → contract tests → data quality → CI → documentation → pull request.
📌 Data, Graph & ML Platform Engineer with Freight MNC (Pune)
🏢 Seventh Contact Hiring Solutions
📍 Pune