Overview

The Data Engineer builds and tests data pipelines in Databricks, covering ingestion, transformation, and Gold-layer modeling, against tasks assigned by the Lead Engineer at the start of each sprint and refined daily. This role suits an early-to-mid career engineer seeking rapid growth. The Data Engineer is expected to ask questions when requirements are unclear and to raise blockers promptly rather than delay progress.

Responsibilities

• Build and test Databricks pipelines (PySpark/SQL, Delta Lake) for ingestion, transformation, and modeling tasks assigned during sprint planning.

• Write and maintain data quality checks and tests for pipelines built.

• Participate in daily standups, reporting progress accurately, asking questions when requirements are unclear, and flagging blockers promptly.

• Follow the platform's data security model (RLS/CLS) and coding/documentation standards set by the Lead and Senior Engineers.

• Support integration testing and reconciliation checks for Boomi-based data flows (e.g., JDE-Concur, JDE-UKG) as assigned.





• Apply feedback from code reviews to subsequent work.

• Document pipelines clearly enough for another engineer to assume ownership.

Qualifications

Required Qualifications

• 2–4 years of data engineering experience, including hands-on work with Spark/PySpark and SQL.

• Hands-on Databricks experience (Delta Lake, notebooks, jobs/workflows).

• Working understanding of data warehousing/lakehouse concepts (Bronze/Silver/Gold or comparable layered modeling).

• Experience working in an agile team with daily standups and sprint-based delivery.

• Robust written and verbal English communication skills.

Preferred Qualifications

• Databricks Certified Data Engineer Associate.

• Exposure to financial or ERP source data — JD Edwards (JDE), EcoSys, or Microsoft Dynamics — or general ledger/cost data.

• Exposure to an EPC, construction, energy, or other asset-heavy industry.

• Familiarity with Unity Catalog-based secu

📌 Data Engineer- Databricks (Hyderabad)
🏢 Annalect, Omnicom Media Group India Private
📍 Hyderabad