Company: Om Tech
Job type: Full-time
Location: Remote (India)
Experience: 2–4 years

About the role

Om Tech is building a data platform that collects structured listings and documents from a large number of public websites, many of them government and public-sector sites, and turns them into clean, reliable, searchable data for our product.

You'll own the data collection layer end to end: building the scrapers, keeping them running every day, making sure the data is accurate and free of duplicates, and deploying it all on our own Linux servers. This is a hands-on role with real ownership. You won't just be writing one-off scripts.

What you'll do

- Build and maintain scrapers for 50+ websites. Some share a common platform, so one well-designed spider can serve many sites through configuration. Others need their own parser.
- Handle the messy parts of real websites: pagination, session-based navigation, JavaScript-rendered pages, slow and unreliable servers, and layouts that change without notice.
- Extract data from PDFs and documents (notices, schedules, tables) as well as HTML pages.
- Design and keep a clean PostgreSQL database: stable primary keys, upserts, de-duplication within and across sources, change history, and data validation.
- Normalise messy data: dates and times, amounts, text in Indian regional languages, and inconsistent categories.
- Make the system reliable: isolated jobs, retries and back-off, resumable runs, detection when a site's layout breaks, logs, monitoring and alerts.
- Measure quality every day: coverage, freshness and field accuracy against a reference dataset. Fix gaps before users notice them.
- Deploy and run it on Linux servers: Docker,



scheduled jobs, backups, and deploys through CI from GitHub.
- Scrape responsibly: public pages only, respecting rate limits and site terms. We don't bypass captchas, logins or access controls.
- Work with our backend team to deliver the data into the main product through a clean API.

Must have

- Strong Python, including clean, tested, maintainable code
- Production experience with Scrapy and/or Playwright (or Selenium)
- Solid understanding of HTML, the DOM, XPath/CSS selectors, HTTP (sessions, cookies, headers) and JSON APIs
- PostgreSQL and SQL beyond the basics: schema design, constraints, indexes, upserts
- Experience keeping scrapers running over time, not just writing them once
- Comfortable on Linux servers: SSH, Docker, cron or schedulers, logs
- Git and pull-request based work

Valuable to have

- PDF and table extraction (pdfplumber, Camelot or similar)
- Redis or another job queue
- De-duplication, record linkage or data-quality work
- Monitoring and alerting setups
- Experience with Indian government or public-sector websites
- Using AI coding tools (Claude Code, Copilot, Cursor) productively
- Basic Django or REST API knowledge

How we work

- You'll work with clear milestones and real ownership of a core part of the product.
- We use AI coding tools heavily and expect you to use them well, while understanding and testing everything you ship.
- Remote, flexible hours, with regular check-ins.

Benefits

- Flexible schedule
- Paid sick time
- Paid time off
- Work from home

Pay: From ₹10,000.00 per month

Benefits:

- Flexible schedule
- Paid sick time
- Paid time off
- Work from home

Work Location: Remote

📌 Data Extraction & Web Scraping Engineer (India)
🏢 OM-TECH
📍 India