Learn more about Sarah's presentation:

Web Scraping at Scale with Quality and Compliance

Building and managing your web scraping infrastructure is a challenging task. But when it comes to scaling your data gathering projects, for example, from hundreds to millions of requests per day, even more difficulties and unexpected challenges appear. This presentation will cover almost every aspect of web scraping at scale – from the technical details to the legal risks.

Sarah will cover a variety of topics, including:  

  • How to scale a web scraping operation with reliability & compliance;

  • What to look for in an automation framework;

  • How to define, measure and track KPIs for a public web data extraction operation to ensure quality;

  • Key compliance concerns & how to mitigate legal risks.