Lead Data Engineer (India)

Lead Data Engineer (India)

24 Oct
|
Firmable
|
India

24 Oct

Firmable

India

Read this first. This role is not for you if:
• You want to manage people more than write code. This role is 80% hands-on engineering, 20% leading the team
• You've never had engineers formally reporting to you. Mentoring or acting as a tech lead without direct reports doesn't count
• You haven't written production code in the last year
• You haven't architected large-scale systems that source data from the web, with proxy rotation, anti-bot handling and recovery. BeautifulSoup scripts or off-the-shelf scraping tools don't count
• You've never run LLMs inside an extraction pipeline with labelled eval sets and versioned prompts
• You haven't built an agent, skill, or tool-calling pipeline that other engineers rely on
• You've followed a team's technical standard but never set one
• Your experience is mostly services or consulting delivery, without long-term ownership of a platform Firmable sells company and people data to sales teams across Asia Pacific and the US, and we're taking it worldwide. Every record we sell starts in a sourcing pipeline, and this role designs it. What you'll work on You'll architect the system that turns the open web into structured company and people data for every market we sell in, and lead the engineers who build on it.
• Extraction at scale: anti-bot defences, JS rendering, proxy and IP strategy, schema drift and recovery
• An extractor framework that lets an engineer ship a recent source in days
• Agentic extraction: rules first, cheap decision models for triage,



stronger LLMs for the hard pages, structured-output validation, and a human-review queue for whatever's left
• Evals, traces and cost ceilings on every model step, with alerts when a vendor changes a model under us
• A skills library of versioned SKILL.md specs that engineers and agents both run Must haves
• 7–10 years building production extraction, ETL or data pipeline systems
• Expert Python, advanced SQL, and Airflow or similar at production scale Nice to have: Snowflake or Redshift
· AWS (Lambda, S3, ECS, Glue)
· embeddings for matching
· Braintrust, Promptfoo or Inspect
· Logfire or OpenTelemetry
· B2B data
· startup experience How we build We write code with coding agents like Claude Code by default, and engineers spend their time on design, review and evals. In production we pick the model for the job: frontier LLMs for messy extraction and reasoning, small typed decision models when the answer is a yes/no, a choice or a score, and MCP servers and agent SDKs when a model needs tools. When something new ships, we run it against our evals that week. Every LLM call is logged with its prompt version, model and cost. Small teams, no layers, weekly releases moving to daily. No fixed hours, no handholding. The deal Competitive base and meaningful equity. The extractor framework, skills library and eval harnesses are mostly unbuilt, so you'll be setting the design. Firmable is an equal opportunity employer. We believe diverse teams build better products.

📌 Lead Data Engineer (India)
🏢 Firmable
📍 India

Reply to this offer

Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.

Subscribe to this job alert:

Get the latest job offers by email for: lead data engineer (india) / india

Subscribe to this job alert:

Get the latest job offers by email for: lead data engineer (india) / india