15 Nov
|
Firmable
|
India
Read this first. This role is not for you if: You want
to manage people more than write code. This role is 80% hands-on
engineering, 20% leading the team You've never had engineers
formally reporting to you. Mentoring or acting as a tech lead
without direct reports doesn't count You haven't written production
code in the last year You haven't architected large-scale systems
that source data from the web , with proxy rotation, anti-bot
handling and recovery. BeautifulSoup scripts or off-the-shelf
scraping tools don't count You've never run LLMs inside an
extraction pipeline with labelled eval sets and versioned prompts
You haven't built an agent, skill, or tool-calling pipeline that
other engineers rely on You've followed a team's technical standard
but never set one Your experience is mostly services or consulting
delivery, without long-term ownership of a platform Firmable sells
company and people data to sales teams across Asia Pacific and the
US, and we're taking it worldwide. Every record we sell starts in a
sourcing pipeline, and this role designs it. What you'll work on
You'll architect the system that turns the open web into structured
company and people data for every market we sell in, and lead the
engineers who build on it. Extraction at scale: anti-bot defences,
JS rendering, proxy and IP strategy, schema drift and recovery An
extractor framework that lets an engineer ship a new source in days
Agentic extraction: rules first, cheap decision models for triage,
stronger LLMs for the hard pages,
structured-output validation, and
a human-review queue for whatever's left Evals, traces and cost
ceilings on every model step, with alerts when a vendor changes a
model under us A skills library of versioned SKILL.md specs that
engineers and agents both run Must haves 7–10 years building
production extraction, ETL or data pipeline systems Expert Python,
advanced SQL, and Airflow or similar at production scale Nice to
have: Snowflake or Redshift · AWS (Lambda, S3, ECS, Glue) ·
embeddings for matching · Braintrust, Promptfoo or Inspect ·
Logfire or OpenTelemetry · B2B data · startup experience How we
build We write code with coding agents like Claude Code by default,
and engineers spend their time on design, review and evals. In
production we pick the model for the job: frontier LLMs for messy
extraction and reasoning, small typed decision models when the
answer is a yes/no, a choice or a score, and MCP servers and agent
SDKs when a model needs tools. When something recent ships, we run it
against our evals that week. Every LLM call is logged with its
prompt version, model and cost. Small teams, no layers, weekly
releases moving to daily. No fixed hours, no handholding. The deal
Competitive base and meaningful equity. The extractor framework,
skills library and eval harnesses are mostly unbuilt, so you'll be
setting the design. Firmable is an equal opportunity employer. We
believe diverse teams build better products.
📌 Lead Data Engineer (India)
🏢 Firmable
📍 India