- Design and build distributed scraping architectures that can scale across thousands of domains.

- Reverse engineer APIs and mobile apps using tools like

Frida

,

mitmproxy

,

Charles Proxy, burp and many more.

- Detect common anti-bot measures (like JavaScript checks or browser behavior detection) to build crawlers that adjust to appear more human-like.

- Lead technical innovation in scraping: identify edge cases, prototype current approaches, and share knowledge with the team.

- Set up anomaly detection and monitoring to detect blockers or failures early

Qualifications

What You Bring

- 7+ years of experience with

Python and large-scale scraping projects.

- Deep understanding of anti-bot technologies and how to bypass them, fingerprinting, WAFs, JavaScript challenges, etc.

- Experience working with

Scrapy

,

Requests

,

httpx

,

Selenium

,

Playwright

, or custom headless browser drivers.

- Skilled at handling proxies (residential, datacenter, rotating), session management, cookie injection,



and TLS tweaking.

- Strong debugging skills across browser, HTTP traffic, and device-level interactions.

- You think independently and enjoy solving problems no one has solved before.

- Familiarity with structured storage formats (Parquet, Avro), or real-time stream processing.

- Good knowledge of databases like MongoDB, InfluxDB, and Redis, including how they work and how to optimize them.

➕ Bonus Skills

- Experience with real-device simulation for mobile scraping.

- Comfortable building and maintaining distributed task queues (Celery, Kafka, etc.).

- Experience scraping high-security platforms or mobile-first environments.

- Exposure to Linux or cloud environments (AWS, GCP, etc.).

- Basic understanding of Kubernetes, Cloud Run, and Cloud Functions to deploy and orchestrate scraping workloads.

📌 Software (Web Scraping, Data acquisition, Data collection) (Pune)
🏢 Nielseniq
📍 Pune