We are seeking an experienced Web Scraping Engineer to data extraction efforts for our enterprise clients. In this role, you will be tasked with creating and maintaining robust, large-scale scraping systems for gathering structured data.
Responsibilities:
Develop and optimize custom web scraping tools and workflows.
Integrate scraping systems with data storage solutions like SQL and NoSQL databases.
Troubleshoot and resolve scraping challenges, including CAPTCHAs, rate limiting, and IP blocking.
Provide technical guidance on scraping best practices and standards.
Skills Required:
Expert in Python and scraping libraries such as Scrapy and BeautifulSoup.
Deep understanding of web scraping techniques and challenges (CAPTCHAs, anti-bot measures).
Experience with cloud platforms (AWS, Google Cloud).
Solid background in databases and data storage systems (SQL, MongoDB).
Skills:- Web Scraping, Web crawling, scrapy and Beautiful Soup