Python Mid/Senior Developer – Web Scraping & Automation
2-5 Years
Ahmedabad
Full-Time
About Us
Actowiz Solutions is a leading provider of data extraction, web scraping, and automation solutions. We empower businesses with actionable insights by delivering clean, structured, and scalable data through cutting-edge technology. Join our fast- growing team and lead projects that shape the future of data intelligence.
Role Overview
We are looking for a skilled Python Web Scraping Engineer with at least 2+ years of hands-on experience to join our technical team. The ideal candidate is not just familiar with Python scripting, but is an expert in the Scrapy framework.
You will be responsible for designing, deploying, and maintaining scalable crawling spiders that harvest data from various public sources. You will work within our existing Scrapy infrastructure to ensure high-volume data extraction is clean, reliable, and resistant to antibot measures.
Key Responsibilities
Design, build, and deploy high-performance spiders using the Scrapy Framework.
Handle JavaScript-heavy websites using headless browsers or middleware.
Implement anti-bot bypass strategies:
Proxy rotation
User-agent spoofing
CAPTCHA solving (manual or automated services)
Cookie/session management
Implement Item Pipelines to clean, validate, and structure scraped data before storage.
Implement strategies to bypass complex bot detection (Cloudflare, Akamai) using Scrapy plugins or by integrating headless browsers (Selenium/Playwright) only when necessary.
Manage effective crawling rates to respect server loads while maximizing data throughput.
Maintain and optimize existing spiders to adapt to changes in target website DOM structures.
Parse and clean raw HTML/JSON/XML data and convert it into structured output formats (CSV, JSON, database entries).
Validate data for accuracy, completeness, and consistency.
Integrate with public/private APIs when available to reduce scraping complexi