Overview

The Senior Data Engineer owns substantial components of the Databricks platform, including full pipelines from ingestion through Gold-layer modeling. This role is expected to take on tasks with unresolved ambiguity, resolve open questions efficiently, and deliver without close supervision. The Senior Data Engineer identifies data and design issues before they affect downstream work and supports less experienced Data Engineers on the pod in developing the same judgment.

Responsibilities

• Design and build Databricks pipelines (Spark/PySpark, Delta Lake, Unity Catalog) across Bronze, Silver, and Gold layers for financial and operational data sets.

• Own a defined workstream (e.g., a source system or a domain such as project financials) from intake through delivery, including testing and documentation.

• Participate actively in sprint planning and daily standups, surfacing risks and clarifying requirements rather than waiting for direction.

• Review code and output from Data Engineers on the pod and provide direct, constructive feedback.





• Implement and validate row-level and column-level security controls on owned data sets.

• Support Boomi-based integration build and validation work (e.g., JDE-Concur, JDE-UKG) as the pod assumes ownership from the onshore Integration & Middleware team.

• Partner with the Lead Engineer to break down larger backlog items into task-ready work.

• Identify and raise technical debt, data quality issues, or design gaps proactively, rather than deferring to QA.

Qualifications

Required Qualifications

• 5+ years of data engineering experience, including at least 2–3 years of hands-on Databricks experience (Spark/PySpark, Delta Lake, Unity Catalog).

• Experience building pipelines from financial or ERP source systems — JD Edwards (JDE) directly preferred, with EcoSys or Microsoft Dynamics also relevant — including general ledger, cost, or project financial data.

• Robust SQL and Python skills, with the ability to own a p

📌 Senior Data Engineer- Databricks (Hyderabad)
🏢 Annalect, Omnicom Media Group India Private
📍 Hyderabad