Job Description
Role Overview
nWe are looking for a Web Scraping Engineer to design, build, and operate high-reliability web data acquisition systems in dynamic web environments. The role is hands-on and technical, focused on solving real-world scraping challenges such as anti-bot defenses, session management, browser automation, and large-scale parallel crawling.
nKey Responsibilities
n Core Web Scraping & Data Acquisition
n
- n
- Design and implement high-parallel crawlers for complex, agile websitesn
- Handle JavaScript-heavy sites using Python / Playwright / Puppeteer / CDPn
- Implement robust session management, cookies, headers, and identity rotationn
- Build resilient pipelines for structured & unstructured data extractionn
- Ensure data accuracy, freshness, and consistency at scalen
nAnti-Bot & Protection Handling
n
- n
- Analyze and mitigate anti-bot protections, including:n
- Cloudflaren
- Akamain
- Qratorn
- ServicePipen
- Variti (and similar)n
- Work with browser-level and network-level fingerprinting:n
- TLS / JA3n
- CDP fingerprintsn
- Header and behavior-based detectionn
- Integrate and optimize CAPTCHA solving strategies (where legally permitted)n
nRequired Skills & Experience
nMust-Have
n
- n
- 1–3+ years of hands-on web scraping / crawling experiencen
- Strong proficiency in Python (Scrapy, asyncio, aiohttp, Playwright)n
- Deep understanding of:n
- HTTP, cookies, headers, redirectsn
- Browser automation & JS executionn
- Experience scraping highly protected websitesn
- Solid debugging skills in hostile scraping environmentsn