This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Data Engineer (Web Scraping) based in India.

This is a senior, hands-on opportunity to own the next generation of reliable web-data acquisition within a modern data platform.
You will design, build, and operate production-grade Python scrapers and scalable ingestion pipelines that support trusted research and data products.
The role focuses on improving the maturity, reliability, and scalability of web-scraping capabilities across a growing data setting.
You will establish reusable patterns for extraction, scheduling, storage, monitoring, validation, and failure handling.
From investigating new data sources to deploying and supporting production workloads, you will have substantial ownership over the full engineering lifecycle.
The role combines deep Python engineering with cloud infrastructure, data pipelines, observability, and practical problem-solving in a remote-first environment.




You will work independently while collaborating with a distributed engineering team through code reviews, documentation, and structured development workflows.

n

Accountabilities:

• Own the development, deployment, and ongoing operation of web-scraping and web-data ingestion pipelines.

• Design and establish scalable web-scraping frameworks with reusable patterns for extraction, scheduling, storage, monitoring, validation, and failure handling.

• Build, maintain, and improve reliable production scrapers for both new and existing data sources.

• Investigate websites and determine the most appropriate acquisition method, including APIs, direct HTTP requests, HTML parsing, browser automation, or third-party tooling.

• Evaluate build-versus-buy options for scraping infrastructure and external services, considering capabilities, reliability, cost, operational complexity, and risk.

• Ensure web-data acquisition activities appropriately account

📌 Senior Data Engineer (India)
🏢 Jobgether
📍 India