Migrate batch data producers and their dependencies to governed AWS data targets. Protect downstream behaviour while making lineage, migration status and replay outcomes visible.

What you will deliver

- Inventory batch jobs, consumers, tables and owners; identify missing lineage and sequence changes around consumer readiness.
- Update pipeline reads and writes for approved roles and governed targets; troubleshoot data access and processing failures alongside platform engineers.
- Migrate replay pipelines in non-production; implement secure retries, idempotent processing, reconciliation and recovery.
- Publish reliable pipeline and dependency status to the migration dashboard; maintain runbooks, tests and evidence for acceptance.

Essential experience

- Strong Python or Scala, SQL and Apache Spark or PySpark, with hands-on batch pipeline development and migration.
- Applied AWS S3, Glue and catalogue experience; understanding of Athena, IAM roles and the implications of governed data access.
- Delivered orchestration, dependency tracking, data quality checks and backfills or replay; can explain duplicate prevention, partial failures and schema changes.

Useful additional experience: Iceberg or Delta Lake,



lineage tooling, CI/CD and infrastructure as code. Experience with an equivalent enterprise batch framework is relevant; knowledge of a customer-specific platform is not required. Early delivery expectations An evidence-backed dependency map and prioritised backlog, followed by a representative migrated pipeline with reconciled outputs, documented recovery and visible status. Milestones depend on discovery and access.

What to include with your CV

Describe a pipeline migration or backfill you delivered, including data volume, downstream dependencies, reconciliation approach, one production issue and your direct contribution.

Include role code, current Indian city, exact earliest start date, interview availability and expected engagement terms. Customer Details are confidential

Pay: Up to ₹3,500,000.00 per year

Benefits

- Health insurance
- Paid time off
- Provident Fund

Application Question(s):
- Current CTC
- Expected CTC
- Notice Period

Location:
- Delhi, Delhi (Delhi) (Preferred)

Work Location: Hybrid remote in Delhi, Delhi (Delhi)

📌 AWS Data Pipeline and Migration Engineer (Delhi)
🏢 Vamstar
📍 Delhi