This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Lead Data Engineer based in India.

As a Lead Data Engineer, you’ll architect and own the end-to-end data sourcing and ETL infrastructure powering a large-scale B2B data platform.
You’ll transform complex, unstructured web data into accurate, structured datasets across multiple international markets.
The role combines advanced data engineering, web extraction, LLM infrastructure, agentic workflows, and production-grade orchestration.
You’ll establish the technical standards and frameworks that enable other engineers to build reliable, scalable extraction systems.
A significant part of the role is hands-on, tackling difficult problems such as anti-bot defenses, JavaScript-heavy sources, schema drift, and cost optimization.
You’ll also shape how AI is evaluated and operated in production through versioning, observability, labelled datasets, and precision and recall measurement.
Working within a lean, senior, AI-native environment,



you’ll have substantial autonomy and direct ownership of architecture, quality, performance, and outcomes.

Accountabilities

-
Architect and own the complete data sourcing and ETL layer, covering extraction, normalization, deduplication, validation, loading, reliability, performance, and cost.

-
Define and maintain standards for data coverage, accuracy, schema design, and the appropriate use of deterministic rules versus LLM-based processing.

-
Design scalable extractor frameworks, abstractions, patterns, and tooling that enable engineers to rapidly build reliable current extraction systems.

-
Develop and ship complex extractors capable of handling anti-bot defenses, proxy and IP rotation, JavaScript-heavy rendering, schema drift, and poorly structured sources.

-
Build agentic extraction workflows incorporating rule-based triage, LLM escalation, structured-output validation, retries, and human-review queues.

-
E

📌 Lead Data Engineer (India)
🏢 Jobgether
📍 India