Data Engineer

Location: Mumbai (Aeroli / Worli)

Function: Technology / Data Engineering Role Purpose

We are looking for a Data Engineer to build and operate production-grade data pipelines on contemporary cloud and lakehouse platforms, supporting core banking, risk, and regulatory reporting data flows. This is a hands-on engineering role — the bank needs someone who can independently design, debug, and optimise pipelines under real production constraints, not just describe the concepts.

Key Responsibilities

Design, build, and maintain scalable ETL/ELT pipelines processing large, multi-source banking datasets (transactions, customer, risk, regulatory).

Architect metadata- and config-driven ingestion frameworks that can onboard new data sources with minimal rework.

Implement and tune Spark/PySpark jobs for performance — partitioning, shuffle optimisation, incremental/CDC loads, and cost efficiency at scale.

Own data quality: build validation, schema-drift handling, and audit/lineage tracking into pipelines, not as an afterthought.

Build and maintain orchestration (Airflow or equivalent) across multi-dependency batch workflows, with SLA monitoring, retry logic, and failure alerting.

Set up and maintain observability (dashboards, logs, alerting) for pipeline health and troubleshoot production incidents independently.

Work within CI/CD pipelines for deployment across environments (dev/UAT/prod), following banking change-management and audit requirements.

Collaborate with data governance, risk, and business teams to translate reporting/regulatory requirements into pipeline design.

Must-Have Skills (Practical, Not Just Theoretical)

Strong hands-on experience with Databricks/Spark/PySpark in a production environment — able to walk through real debugging scenarios (job failures,



data skew, schema mismatches, late-arriving data) with specifics, not textbook answers.

Delta Lake or equivalent lakehouse experience — incremental processing, ACID handling, time travel, optimisation strategies.

Strong SQL and Python for data engineering (not just scripting — able to reason about performance trade-offs).

Cloud data engineering on AWS (S3, Glue, Lambda, EMR) or Azure equivalents.

Orchestration tooling (Airflow or similar) — designing DAGs for real dependency chains, not simple linear jobs.

Experience with data quality/governance tooling (e.g. Collibra, Unity Catalog, or equivalent) and audit/lineage requirements typical in regulated environments.

Comfortable being tested on scenario-based, whiteboard-style problems — e.g., "pipeline is failing intermittently in prod, walk me through your diagnosis," or "design an ingestion framework for 10 new data sources with different schemas."

Good to Have

Prior experience in BFSI / regulated environments (data lineage, audit trail, compliance-driven data handling).

Exposure to observability stacks (Grafana, Kibana, Prometheus, OpenSearch).

CI/CD experience (Jenkins, GitHub Actions) in a banking change-control context.

Experience Level

2–5 years of hands-on data engineering experience, with at least one production system owned end-to-end (not just contributed to). Candidates should be prepared to discuss specific incidents, design trade-offs, and performance-tuning decisions in depth.

Screening Note (Internal)

Prior interview feedback on a similar profile flagged strong theoretical/communication skills but a gap in scenario-based practical depth. Screening for this role should weight hands-on, whiteboard-style problem-solving heavily — not just tool/keyword familiarity on the CV.

📌 Data Engineer_Mumbai_Immediate Joiner_12 months contract (Mumbai)
🏢 BDO India
📍 Mumbai