About Forage AI: Forage AI delivers large-scale data collection and processing platforms, including web crawlers, document parsers, data pipelines, AI/Agentic Solutions, and AI-assisted workflows.
Our primary programming language is Python. We design both cloud-native and cloud-agnostic solutions, primarily on AWS, while also working with GCP and Azure. We value high ownership, robust collaboration, and pragmatic, well-documented engineering.

Role Overview: We are looking for a hands-on Engineering Lead who combines deep Python expertise with proven experience building high-scale web scraping and data collection systems. You will own the full lifecycle of our data acquisition platform — from architecting resilient crawlers and parsers to leading a team of engineers and ensuring production reliability. Cloud/DevOps
experience is valued but not the primary focus; your core strength is in Python craftsmanship
and data extraction at scale.

Key Responsibilities

Engineering Leadership
1. Own end-to-end delivery across requirements, design, implementation, testing,



and production operations.
2. Lead and grow a team of engineers through mentorship, code reviews, pairing on hard problems, and raising overall engineering quality.
3. Translate business and client needs into clear technical plans; manage risks, trade-offs, and timelines.
4. Establish and enforce engineering best practices: branching strategy, code standards, testing discipline, and incident/RCA processes.

Python & Scraping Excellence
5. Architect and build scalable, fault-tolerant crawling and parsing systems (Scrapy, Selenium, Playwright, or equivalent frameworks).
6. Write production-grade Python and set the bar via reference implementations, design docs, and thorough code reviews.
7. Build robust mechanisms for anti-bot evasion, proxy rotation, rate limiting, and content fingerprinting.
8. Design parsers and extractors for structured and unstructured content across diverse and changing web targets.

Data

📌 Assistant Lead (India)
🏢 Forage AI
📍 India