Role & responsibilities

We are looking for a passionate Scraper Developer to build and maintain web scraping solutions that support our competitive intelligence platform. The ideal candidate enjoys analyzing websites, identifying data sources, and developing robust extraction pipelines to collect product and pricing information at scale.

Key Responsibilities
Develop and maintain web scrapers for competitor websites.
Extract structured data including product details, pricing, availability, categories, and URLs.
Analyze websites using Chrome DevTools to identify HTML, API, AJAX, and JavaScript-rendered data sources.
Build scraping solutions using Scrapy, BeautifulSoup, lxml, requests, Playwright, and Selenium.
Handle pagination, agile content, API integrations, and data normalization.
Write clean, maintainable, and well-documented Python code.
Create and execute unit tests using pytest to ensure data accuracy and reliability.




Troubleshoot and fix scraper issues caused by website structure changes.
Collaborate with QA and engineering teams to resolve data quality issues.

Required Skills
Robust experience with Python
Hands-on experience with Scrapy, BeautifulSoup4, lxml, and requests
Positive knowledge of XPath, CSS Selectors, and JSON parsing
Experience with Playwright and/or Selenium
Familiarity with Chrome DevTools (Network & Elements tabs)
Knowledge of pytest, Git, and debugging techniques
Understanding of web technologies including HTML, CSS, JavaScript, and APIs

Preferred Skills
Experience with anti-bot techniques, proxy rotation, and cookie handling
Knowledge of CAPTCHA-solving integrations
Experience scraping e-commerce, retail, travel, or comparison-shopping websites
Basic understanding of SQL and Docker
Active GitHub profile showcasing scraping projects

📌 Scraper Developer Swiss Based Mnc Co Bengaluru
🏢 Recruise India Consulting
📍 Bengaluru