Responsibilities

- Assist with the design and implementation of advanced scraping solutions using Python, Playwright, and AWS services.
- Automate interactions with JavaScript-heavy and authentication-secured websites, including handling MFA, CAPTCHAs, and session or token-based login flows.
- Architect scraping pipelines using serverless AWS components such as Lambda, Step Functions, S3, CloudWatch, and Secrets Manager.
- Build systems that scale to support high volumes of data extraction with fault tolerance, retries, and intelligent logging.
- Integrate and manage complex workflows across multiple portals, APIs, and data sources.
- Contribute to architectural decisions, tooling, and best practices.
- Responsible for Code reviews and deployments.

Qualifications

- Bachelor’s or master’s degree in computer science, software engineering, or related field, or equivalent skilled experience
- 10+ years of experience in Python development, with a strong focus on automation and data extraction
- Proven expertise in web scraping using tools like Playwright, Selenium, Scrapy, BeautifulSoup, and requests
- In-depth experience handling multi-step authentication flows,



including multi-factor authentication (MFA), CAPTCHA solving, and session or cookie management
- Proficient in deploying and managing scraping workloads in AWS, particularly Lambda, S3, IAM, CloudWatch, and Secrets Manager
- Experience with asynchronous programming, headless browsers, and JavaScript-rendered content
- Solid understanding of web protocols (i.e. HTTP, HTTPS, cookies, headers), and the ability to reverse-engineer network calls and authentication mechanisms
- Comfortable working with APIs, JSON/XML, and data transformation
- Experience with CI or CD pipelines, Docker, and infrastructure as code (i.e. CloudFormation, Terraform)
- Familiar with data pipeline orchestration tools such as Apache Airflow or AWS Step Functions
- Prior experience scraping from secure or enterprise-level portals (i.e. financial, healthcare, legal)
- Background in data engineering or ETL workflows is a plus
- Exposure to Python testing frameworks and writing unit or integration tests

Skills:- Python and Django

📌 Lead Python Developer (Jaipur)
🏢 Resources Valley
📍 Jaipur