Job DescriptionPython & JavaScript Developer – AI & Web Scraping
NExperience: 4-7 Years
NLocation: Remote
NMode of Engagement: Full-time
NNo of Positions: 3
NEducational Qualifications: Bachelor's degree in computer science, Information Technology
NIndustry: IT / Software Development
NNotice Period: Immediate Joiners Preferred
NAbout the Role
nWeare looking for a hands-on Web Scraping / Crawling Engineer with 3–5 years of experience in web scraping, browser automation, and scalable data extraction.
NThe ideal candidate should have solid experience working with dynamic and JavaScript-heavy websites, building reliable crawling workflows, handling crawling failures, and working with distributed processing systems.
NThis is a highly technical, hands-on role. You will be expected to design, develop, debug, optimize, and maintain web crawling and data extraction systems.
NResponsibilities
n
- N
- Design, develop, and maintain scalable web crawling and scraping systems for dynamic and JavaScript-heavy websites. N




- Develop browser automation workflows using Playwright, Selenium, Puppeteer, or similar frameworks. N
- Investigate and resolvecrawling issues such as 403/429 responses, redirects, timeouts, rendering failures, session issues, and anti-bot challenges. N
- Develop robust retry, fallback, and failure-handling mechanisms to improve crawler reliability. N
- Work with cookies, sessions, browser contexts, headers, proxies, and related crawling mechanisms to maintain state and improve crawling reliability. N
- Build and optimize concurrent and distributed scraping workflows using asynchronous processing, queues, and worker-based architectures. N
- Design data pipelines covering URL processing, crawling, extraction, validation, transformation, and storage. N
- Optimize crawler performance, including concurrency, browser lifecycle, resource utilization, timeouts, and request handling. N
- Implement monitoring and observability for crawl

📌 Web Scraping Engineer-Python (Bengaluru)
🏢 AIMLEAP
📍 Bengaluru