Job Description

About the Role
We are looking for a Senior Software Engineer to join our engineering team building a high-scale, API-first SaaS platform. Our platform processes millions of transactions every month and extracts data from hundreds of external websites and data sources through automated web scraping and integrations. You will own features end to end, from design through production. You will build resilient scraping systems that hold up as source websites change, and you will help raise the engineering bar through code reviews, mentoring, and strong technical documentation.

Our Tech Stack
C# and .NET (latest versions), RESTful APIs, and relational databases. For browser automation and data extraction we use Selenium, Playwright, and HtmlAgilityPack. We run on AWS (Lambda, SQS, S3, ElastiCache, RDS) on serverless and managed infrastructure.

Key Responsibilities

- Own the planning, design, and implementation of complex product features and platform improvements end to end
- Design, build, and maintain web scraping and crawling solutions that extract structured data from hundreds of external websites, including dynamic, JavaScript-heavy sites
- Build scrapers that are resilient to website layout changes, with monitoring that detects breakages quickly and processes to fix them
- Handle scale and reliability in data extraction, including rate limiting, retries, proxy management, session handling, and queue-based processing
- Validate, clean, and normalize extracted data so it is accurate and consistent before it reaches downstream systems
- Make sure scraping is done responsibly, in line with source terms of use and applicable guidelines




- Design, build, and evolve RESTful APIs, making sound design decisions and maintaining backward compatibility
- Write clean, maintainable, and scalable code, and safeguard code quality through pull request reviews
- Create and maintain technical design documents covering design decisions, trade-offs, and operational considerations
- Work closely with Product, Engineering leads, Support, and Implementation teams to clarify requirements and deliver reliable solutions
- Lead troubleshooting of production issues in real time, perform root-cause analysis, and drive post-incident improvements
- Contribute to architectural improvements that improve scalability, performance, security, and observability
- Mentor junior and mid-level engineers through code reviews, pairing, feedback, and knowledge sharing
- Participate in on-call rotations

Required Skills and Experience

- 5+ years of experience building and operating web-based or SaaS products at scale
- Strong hands-on experience with C# and .NET, and a solid grasp of object-oriented principles and design patterns
- 2+ years of hands-on web scraping experience in production, using tools such as Selenium, Playwright, PuppeteerSharp, HtmlAgilityPack, or AngleSharp
- Strong command of HTML/DOM parsing, XPath,



CSS selectors, and regular expressions
- Experience scraping dynamic websites, including headless browsers, handling logins and sessions, and working with pagination, forms, and file downloads
- Experience building reliable, high-volume scraping pipelines, including retry logic, rate limiting, error handling, and monitoring for scraper breakages
- Proven experience designing, building, and maintaining high-load RESTful or GraphQL APIs
- Strong experience with relational databases (SQL Server, PostgreSQL, or MySQL), including schema design, query optimization, and performance tuning
- Experience supporting high-availability production environments and diagnosing customer-impacting issues
- Experience using AI/agentic coding assistants to improve productivity and code quality
- Prior experience mentoring engineers or acting as a technical lead on projects

Good to Have

- Experience with Python scraping frameworks such as Scrapy or BeautifulSoup
- Experience with proxy rotation services and managing large-scale crawling infrastructure
- Hands-on experience with AWS services such as Lambda, SQS, S3, ElastiCache, and RDS
- Experience designing or maintaining CI/CD pipelines
- Experience with Docker and Kubernetes in production

What We Look For

- Strong critical thinking and attention to detail
- Clear communication
- An ownership mindset and a bias for action: you spot problems and drive them to completion with minimal direction
- A humble, team-oriented, self-aware approach, and openness to feedback
- Staying current with industry trends

📌 Senior Software Engineer (India)
🏢 Harphil Software Solutions
📍 India