Role OverviewWe are looking for a Web Scraping Engineer to design, build, and operate high-reliability web data acquisition systems in energetic web environments. The role is hands-on and technical, focused on solving real-world scraping challenges such as anti-bot defenses, session management, browser automation, and large-scale parallel crawling.Key ResponsibilitiesCore Web Scraping & Data AcquisitionDesign and implement high-parallel crawlers for complex, dynamic websitesHandle JavaScript-heavy sites using Python / Playwright / Puppeteer / CDPImplement robust session management, cookies, headers, and identity rotationBuild resilient pipelines for structured & unstructured data extractionEnsure data accuracy, freshness, and consistency at scaleAnti-Bot & Protection HandlingAnalyze and mitigate anti-bot protections, including:CloudflareAkamaiQratorServicePipeVariti (and similar)Work with browser-level and network-level fingerprinting:TLS / JA3CDP fingerprintsHeader and behavior-based detectionIntegrate and optimize CAPTCHA solving strategies (where legally permitted)Required Skills & ExperienceMust-Have1–3+ years of hands-on web scraping / crawling experienceStrong proficiency in Python (Scrapy, asyncio, aiohttp, Playwright)Deep understanding of:HTTP, cookies, headers,



redirectsBrowser automation & JS executionExperience scraping highly protected websitesSolid debugging skills in hostile scraping environmentsNice-to-Have (Not Mandatory)Familiarity with:Kubernetes / Helm (user-level, not platform owner)CI/CD pipelinesDistributed task queues (Redis, RabbitMQ, Kafka)Experience mentoring junior engineersRole DetailsRole: Data EngineerIndustry Type: IT Services & ConsultingDepartment: Engineering – Software & QAEmployment Type: Full Time, PermanentRole Category: Software DevelopmentEducationUG: B.Tech / B.E. in Any SpecializationKey SkillsScrapyWeb CrawlingPythonData ScrapingData ExtractionAbout CompanyKrawlNet Technologies enables advertisers and publishers to run affiliate programs efficiently and at scale. We aggregate product data from a wide range of retailers, allowing publishers and analytics partners to seamlessly access high-quality, normalised data to drive growth and performance.At the core of our platform is web-scale crawling and data extraction, built to handle complex, dynamic web environments. Our mission is to solve critical industry challenges by delivering reliable services for data acquisition, cleansing, normalisation, and enrichment, transforming raw web content into actionable business intelligence.Company InfoAddress: Suratwala Mark Plazzo, Hinjawadi, Pune, Maharashtra 411057

📌 Web Scraping Engineer In Pune (Maharashtra)
🏢 Krawlnet Technologies
📍 Maharashtra