We are looking for a Web Scraping Engineer to design, build, and operate high-reliability web data acquisition systems in energetic web environments. The role is hands-on and technical, focused on solving real-world scraping challenges such as anti-bot defenses, session management, browser automation, and large-scale parallel crawling.
Key Responsibilities
Core Web Scraping & Data Acquisition
Design and implement high-parallel crawlers for complex, agile websites
Handle JavaScript-heavy sites using Python / Playwright / Puppeteer / CDP
Implement robust session management, cookies, headers, and identity rotation
Build resilient pipelines for structured & unstructured data extraction
Ensure data accuracy, freshness, and consistency at scale
Anti-Bot & Protection Handling
Analyze and mitigate anti-bot protections, including:
Cloudflare
Akamai
Qrator
ServicePipe
Variti (and similar)
Work with browser-level and network-level fingerprinting:
TLS / JA3
CDP fingerprints
Header and behavior-based detection
Integrate and optimize CAPTCHA solving strategies (where legally permitted)
Required Skills & Experience
Must-Have
1–3+ years of hands-on web scraping / crawling experience
Strong proficiency in Python (Scrapy, asyncio, aiohttp, Playwright)
Deep understanding of:
HTTP, cookies, headers, redirects
Browser automation & JS execution
Experience scraping highly protected websites
Solid debugging skills in hostile scraping settings