About Chryselys
Chryselys is a Great Place to Work Certified Pharma Analytics & Business consulting company that delivers data-driven insights leveraging AI-powered, cloud-native platforms to achieve high-impact transformations. We specialize in digital technologies and advanced data science techniques that provide strategic and operational insights.
Role Summary
Own and extend a production-bound POC that combines resilient web-crawling engineering with an AWS Bedrock-based LLM verification agent, in a healthcare-data-compliance-sensitive context.
Responsibilities
Replace scraped SERPs with a paid search API behind the existing search_domain() interface.
Move fetch.py to async httpx with per-domain concurrency limits and politeness budgets.
Introduce proxy rotation and egress management; retire the single-IP failure mode.
Make failure loud: typed outcomes, structured logs, metrics, drift and volume alerts.
Reverse-engineer payer XHR/JSON endpoints to replace Playwright recipes wherever possible.
Honor robots.txt Disallow and Crawl-delay; build a per-domain ToS and licensing register.




Containerize and schedule the pipeline; add CI running the offline tests on every change.
Add HAR replay and golden-file parser tests on real payer HTML, plus daily canary crawls.
Skills
Web & protocol fundamentals — HTTP/1.1, HTTP/2, HTTP/3, TLS, JA3/JA4 fingerprinting, cookies/sessions, OAuth2/JWT/SSO, DOM, encodings [Must]
Legacy stack (real mileage) — urllib, requests, mechanize, cURL/wget, BeautifulSoup, lxml, html5lib, XPath/XSLT, Scrapy (middlewares, pipelines, CrawlSpider), Selenium 3 patterns, Splash/PhantomJS, RSS/Atom/SOAP/XML feeds, ASP.NET VIEWSTATE postbacks, frame & table-layout scraping, sitemap.xml, WARC/Common Crawl [Must]
Contemporary stack — Playwright (contexts, tracing, network interception), Puppeteer, Selenium 4/CDP, httpx/aiohttp + asyncio, curl_cffi TLS impersonation, selectolax, scrapy-playwright, Crawlee/Apify, JSON-LD/microdata harvesting [Must]
Reverse engineerin

📌 Subject Matter Expert (Hyderabad)
🏢 Proof-of-Skill
📍 Hyderabad