Role & responsibilities

We are looking for a passionate Scraper Developer to build and maintain web scraping solutions that support our competitive intelligence platform. The ideal candidate enjoys analyzing websites, identifying data sources, and developing robust extraction pipelines to collect product and pricing information at scale.

Key Responsibilities

- Develop and maintain web scrapers for competitor websites.
- Extract structured data including product details, pricing, availability, categories, and URLs.
- Analyze websites using Chrome DevTools to identify HTML, API, AJAX, and JavaScript-rendered data sources.
- Build scraping solutions using Scrapy, BeautifulSoup, lxml, requests, Playwright, and Selenium.
- Handle pagination, dynamic content, API integrations, and data normalization.
- Write clean, maintainable, and well-documented Python code.
- Create and execute unit tests using pytest to ensure data accuracy and reliability.




- Troubleshoot and fix scraper issues caused by website structure changes.
- Collaborate with QA and engineering teams to resolve data quality issues.

Required Skills

- Strong experience with Python
- Hands-on experience with Scrapy, BeautifulSoup4, lxml, and requests
- Positive knowledge of XPath, CSS Selectors, and JSON parsing
- Experience with Playwright and/or Selenium
- Familiarity with Chrome DevTools (Network & Elements tabs)
- Knowledge of pytest, Git, and debugging techniques
- Understanding of web technologies including HTML, CSS, JavaScript, and APIs

Preferred Skills

- Experience with anti-bot techniques, proxy rotation, and cookie handling
- Knowledge of CAPTCHA-solving integrations
- Experience scraping e-commerce, retail, travel, or comparison-shopping websites
- Basic understanding of SQL and Docker
- Active GitHub profile showcasing scraping projects

📌 Scraper Developer (Swiss Based MNC Co.) (Bengaluru)
🏢 Recruise India Consulting
📍 Bengaluru