Role Overview We are looking for a Web Scraping Engineer to design, build, and operate high-reliability web data acquisition systems in dynamic web environments. The role is hands-on and technical, focused on solving real-world scraping challenges such as anti-bot defenses, session management, browser automation, and large-scale parallel crawling. Key Responsibilities Core Web Scraping & Data Acquisition Design and implement high-parallel crawlers for complex, dynamic websites Handle JavaScript-heavy sites using Python / Playwright / Puppeteer / CDP Implement robust session management, cookies, headers, and identity rotation Build resilient pipelines for structured & unstructured data extraction Ensure data accuracy, freshness, and consistency at scale Anti-Bot & Protection Handling Analyze and mitigate anti-bot protections, including: Cloudflare Akamai Qrator ServicePipe Variti (and similar) Work with browser-level and network-level fingerprinting: TLS / JA3 CDP fingerprints Header and behavior-based detection Integrate and optimize CAPTCHA solving strategies (where legally permitted)



Required Skills & Experience Must-Have 1–3 years of hands-on web scraping / crawling experience Robust proficiency in Python (Scrapy, asyncio, aiohttp, Playwright) Deep understanding of: HTTP, cookies, headers, redirects Browser automation & JS execution Experience scraping highly protected websites Solid debugging skills in hostile scraping environments Nice-to-Have (Not Mandatory) Familiarity with: Kubernetes / Helm (user-level, not platform owner) CI/CD pipelines Distributed task queues (Redis, RabbitMQ, Kafka) Experience mentoring junior engineers Role Details Role: Data Engineer Industry Type: IT Services & Consulting Department: Engineering – Software & QA Employment Type: Full Time, Permanent Role Category: Software Development Education UG: B.Tech / B.E. in Any Specialization Key Skills Scrapy Web Crawling Python Data Scraping Data Extraction About Company KrawlNet Technologie

📌 Web Scraping Engineer in Pune (Maharashtra)
🏢 Krawlnet Technologies
📍 Maharashtra