Role & responsibilities

- Build web scrapers for supplier, wholesaler, and competitor websites
- Extract product names, prices, SKUs, images, descriptions, weights, brands, stock, ingredients, and categories
- Handle energetic websites, pagination, login flows, and anti-bot restrictions lawfully
- Build repeatable ETL/ELT pipelines
- Clean and standardise inconsistent product data
- Match duplicate products across different suppliers
- Map products into Yariyos category structure
- Use AI for product classification, translation, description generation, and attribute extraction
- Store data in a structured database
- Push products into Yariyo through APIs or scheduled imports
- Monitor failed jobs, missing fields, price changes, and stock changes
- Maintain logs, retry logic, validation rules, and data-quality dashboards

Preferred candidate profile





- 2-3 years experience with Python
- Scrapy, Beautiful Soup, or Selenium
- REST APIs and webhooks
- SQL and PostgreSQL
- Airflow, Prefect, Dagster, or similar orchestration
- Docker and Git
- Cloud deployment on AWS, GCP, or Azure
- Proxy, rate-limit, and scraping reliability knowledge
- Data deduplication and entity matching
- Experience with e-commerce product data is highly valuable

Perks and benefits

- Competitive salary based on technical skills and experience
- Long-term, stable remote position with a Germany-based company
- Performance bonus based on pipeline reliability, automation, data quality, and delivery

📌 Data Engineer - Web Scraping and Product Data Pipelines (India)
🏢 Primelandlords
📍 India