Monitoring Web Scrapers: Best Practices

Web scraping for developers, presentation
25 August, 2:40PM BST, 30min

Maintaining web scrapers is a challenging task and constantly monitoring them is an essential part of the maintenance. If (or even when) something goes wrong, it’s important to react fast. Otherwise, consequences can be very expensive.

The challenge grows even bigger when monitoring multiple web scrapers that gather public data from multiple sources. Load management is one of the key factors that must be given proper attention, which leads to even more monitoring.

During this session, Oxylabs Data Analyst Andrius Kūkšta will cover the best practices for building monitoring systems that allow reacting promptly to any potential deviations.

Join this session, if you:

Are considering building your own web scraper
Run web scraping projects
Monitor one on multiple web scrapers

Read About OxyCon Day One

Please note: The views expressed by speakers or moderators are those of the speaker or moderators and not, necessarily, of Oxylabs or other respective organizations. Before engaging in scraping activities of any kind, you should consult your legal advisors.

Keep up with the future of web scraping

Read About OxyCon

Meet the speaker

Andrius Kūkšta, Data Analyst at Oxylabs

Andrius has been working at Oxylabs for nearly four years. During this time, he has worked on various projects and with different products. Most of the projects he worked on were related to web scraping and allowed Andrius to gain hands-on experience building and monitoring web scrapers.

Now, Andrius mainly focuses on applying Artificial Intelligence and Machine Learning technologies to Oxylabs scraping procedures and other areas in the company.

Andrius is passionate about sports, especially - basketball. He also spends his time playing computer games and poker.