Role Overview:

HFG is building its Hyderabad Global Capability Centre (GCC) as the long-term engineering backbone of its global data platform. This is a permanent, full-time role requiring 35 years' total experience with 2+ years in production Databricks, working a minimum four-hour overlap with either CET (approx. 09:0013:00 IST) or US Eastern/Central hours (approx. 18:0022:00 IST) across a 40 hour week. We are looking for Medior Data Engineers to join as early members of the GCC team engineers who are solid on Databricks fundamentals, thrive on BAU pipeline operations and reliability work, and are ready to grow into senior responsibilities within a world-class global engineering environment. Working closely with senior engineers in the UK and Netherlands, the Medior Data Engineer builds hands-on delivery experience while contributing to the day-to-day reliability of the India GCC's data platform. While the core responsibilities of this role are consistent across the organisation, the specific focus, priorities, and day-to-day activities will evolve as the GCC scales. Further detail will be provided during the recruitment and screening process.

About HFG & the Data Platform:

Head First Global (HFG) is a global staffing and workforce solutions company operating across the EU, UK, North America, and APAC. The Global Data & AI team is building the Headless Data Architecture (HDA) a hub-and-spoke federated Lakehouse platform on Azure Databricks, enabling analytics, AI, and operational intelligence across the group's global business units. The platform runs on Azure Databricks with Unity Catalog, Lakeflow, Mosaic AI, and Agent Bricks as core components, and integrates with Snap Logic for enterprise connectivity. All AI workloads route through the Unity AI Gateway. The India GCC team is being built as the long-term engineering backbone of this platform, working in close collaboration with senior engineers in the UK, Netherlands, and the US.

Key Responsibilities:

- Build, maintain, and improve production Databricks data pipelines across the bronze, silver, and gold medallion layers under the guidance of senior engineers.
- Handle BAU (business as usual)



pipeline operations monitoring pipeline health, resolving failures, investigating data quality issues from root cause to fix, and maintaining Delta table health.
- Implement data quality checks, schema validation, and alerting within Databricks Workflows and Delta Live Tables.
- Maintain and extend ADF metadata-driven ingestion pipelines for new source system onboarding.
- Write reliable, idempotent PySpark and Spark SQL transformations following HFG FITT pipeline standards.
- Deploy pipeline changes across environments using Databricks Asset Bundles (DABs) or Azure Dev Ops CI/CD pipelines.
- Document pipeline logic, data lineage, and operational runbooks to keep the platform maintainable as the team scales.
- Collaborate daily with senior engineers and the global team on sprint tasks, code reviews, and knowledge sharing.
- Support the onboarding of new data sources into the Lakehouse mapping source schemas, building ingestion logic, and validating output quality. Contribute to Delta table maintenance OPTIMIZE, VACUUM, and Z-ordering strategies for production table health.

Qualifications & Experience:

Must-have:

- 2+ years of hands-on Databricks experience PySpark, Delta Lake, and Databricks Workflows in production.
- Understanding of medallion architecture (Bronze, Silver, Gold) and ability to work within an established layer pattern.
- Solid Python and SQL for data transformation tasks.
- Azure Data Factory (ADF) pipeline development and monitoring.
- Azure cloud familiarity ADLS Gen2, Key Vault, Azure Dev Ops.
- Good debugging and root-cause analysis skills for production data pipeline incidents.
- Ability to work independently on well-defined tasks and escalate blockers clearly.
- Databricks Certified Data Engineer Associate certification (or active pursuit).




- Snap Logic or comparable iPaaS tool operating and maintaining existing enterprise integration pipelines.
- Azure Synapse Analytics ability to support and monitor existing Synapse workloads during the ongoing migration to Databricks.
- Strong written English for async international collaboration.

Nice to have:

- Unity Catalog exposure schema management, access control basics.
- Databricks Asset Bundles (DABs) for CI/CD deployment.
- Delta Live Tables (DLT) for declarative pipeline development.
- dbt for SQL-layer transformation.
- Experience with Snap Logic, Mule Soft, or similar iPaaS tools.
- Data quality frameworks such as Excellent Expectations or dbt tests.

Working Model & Timezone:

This role is based in Hyderabad and works in a hybrid model. Daily collaboration with HFG teams requires a minimum four-hour overlap with either CET (approx. 09:0013:00 IST) or US Eastern/Central hours (approx.18:0022:00 IST) across a 40-hour week. Candidates should be comfortable with structured asynchronous communication for the remaining hours. The India GCC team meets daily with the global Data & AI team for standups, code reviews, and delivery planning. Strong written and spoken English communication is essential.

Our Platform Stack:

- Azure Databricks Delta Lake, Unity Catalog, DLT, DABs, Lakeflow, Mosaic AI
- Snap Logic enterprise integration and data orchestration
- Azure (ADLS Gen2, Entra ID, Key Vault, Azure Dev Ops, Git Hub Actions)
- Python, PySpark, Spark SQL
- Unity AI Gateway governed LLM routing and observability
- MLflow, Databricks Vector Search, Agent Bricks

What We Offer:

- Be a founding member of the India GCC shape how the team is built and the engineering standards it runs on
- Work on a globally scaled platform serving EU, UK, US, and APAC business units simultaneously
- Direct exposure to AI-enabled data engineering on one of the most modern Databricks deployments in the staffing industry
- Collaborate with senior engineers across the UK and Netherlands who will actively invest in your growth
- Hyderabad-based with a global engineering mandate not an offshore support function

📌 Data Engineer (India)
🏢 Guidant Global India
📍 India