We are looking for an experienced Senior Data Engineer to lead the design, development and operation of enterprise-scale data platforms using Azure Databricks.

The role involves building secure, scalable and auditable data pipelines, managing large-scale data platforms and ensuring high standards of data governance, quality and operational excellence.

The position also involves collaborating with Product, Engineering, AI and Infrastructure teams to deliver reliable data solutions for global financial services environments.

Experience: 10+ Years

Location: Gurugram

Work Model: Hybrid 2 Days Work from Office

Preference: Candidates comfortable with a hybrid work arrangement are strongly preferred.

Key Responsibilities

- Lead end-to-end data pipelines, including secure file ingestion (SFTP), landing, historical storage (Delta) and curated data layers.
- Own Azure Databricks delivery, including notebooks, Workflows/Jobs, Asset Bundles, parameters, concurrency and retries.
- Implement medallion architecture (Landing Staging Curated Analytics) with partitioning, checksums and metadata sidecars.
- Maintain end-to-end pipeline auditability using correlation IDs, trigger records and completion records.
- Partner with SRE teams on data lake/SFTP storage, identity management and secrets for Databricks and related services.
- Implement data quality controls, schema evolution, lineage,



cost optimization and runbooks for pipeline incidents.
- Provide curated, quality-assured datasets from data lakes and SQL warehouses to Product and AI teams for RAG and embedding workflows.
- Ensure data contracts cover schema, metadata, access classifications and refresh frequency. Ownership of RAG corpora and vector stores remains with the respective Product and AI teams.

Must-Have Experience

- 10+ years of data engineering experience, including ownership of production data platforms end to end.
- Solid expertise in Azure Databricks, including Spark, Delta Lake, Workflows/Jobs and Asset Bundles.
- Hands-on experience in data lake architecture and SFTP/file-landing ingestion patterns.
- Strong SQL and Python skills.
- Experience with data orchestration and CI/CD for data engineering code.
- Strong understanding of data governance, security isolation, RBAC, auditability and PII handling.
- Experience with investment data and data modelling.

Nice to Have

- Familiarity with RAG and vector-search consumers (building these systems is not required).
- Experience with dbt, Delta Live Tables, Event Hubs or Change Data Capture (CDC).
- Exposure to Databricks SQL and BI serving patterns.

Preferred Domain Experience Experience in one or more of the following domains is preferred:

- Capital Markets
- Investment Banking
- Asset Management
- Financial Services

📌 Senior Data Engineer (Gurugram)
🏢 Intelaltus
📍 Gurugram