Less than a week ago, Black Friday madness took over the world, and for many, it was an excellent opportunity to grab irresistible money-saving deals. To paint the picture, Black Friday in the USA took the lead as the busiest day for online shoppers (93.2M), leaving previous historical frontrunner day – Cyber Monday (83.3M) behind. Impressive numbers, isn’t it?
To capture the lion’s share of sales, many companies turn to price tracking and product catalog monitoring practices to unlock a further competitive advantage over their direct rivals. Consequently, businesses timely adjust their pricing tactics and stock, which allows them to win the customer. And, to gather this knowledge, companies increasingly rely on web scraping practice, which is usually supported by a proxy infrastructure.
In this article, we will reveal 2019 Black Friday web scraping insights, supported by actual usage statistics of our clients, that utilized Residential and Data center proxies, and our Real-Time Crawler data gathering solution. Also, a special thanks go to our Research Department for providing these valuable insights! Let’s begin.
* Please note: Traffic volumes and request amounts are summarized into trend graphs, ensuring absolute anonymity. The findings don’t include usage statistics of the clients that prefer for their usage statistics to not be recorded.
Residential proxies traffic insights
A data analysis conducted by our Research Department revealed a noteworthy surge in traffic volume via Residential proxies on Black Friday (29th of November), as it can be observed from a noticeable spike in the graph below:
To put this into perspective, let’s compare this year’s Black Friday Residential proxies traffic volume with historical traffic volume metrics:
- In comparison to the previous 7 days usual traffic volume, on the Black Friday, Residential proxies traffic volume increased by a significant 20%.
- When compared to the last 30 days’ usual traffic volume, on the Black Friday, Residential proxies traffic volume increased similarly, by 21%.
- Whereas, in contrast to 2018 Black Friday’s Residential proxies traffic volume, this year’s Black Friday Residential proxies traffic grew by a substantial 65%.
Real-Time Crawler requests volume insights
A data analysis carried out by our Research Department has identified a surge in the volume of requests processed via our Real-Time Crawler on the Black Friday. To paint the picture, here are the comparisons to historical Real-Time Crawler requests volume vs. 2019 Black Friday requests volume:
- In comparison to the previous 7 days usual request volume, on the Black Friday Real-Time Crawler noted a 16%request volume growth.
- When compared to the previous 30 days request volume metrics, on the Black Friday requests volume, in fact, declined by 3%.
- Lastly, in comparison to last year’s Black Friday Real-Time Crawler requests volume, 2019 Black Friday’s request volume grew by a significant 19%.
As you might be aware, our Real-Time Crawler is a robust web data collection tool, optimized to capture data from search engines and e-commerce websites with 100% data delivery success rates. That said, our Research Department has also revealed a substantial requests growth to one of the most prominent marketplaces via the Real-Time Crawler solution on this year’s Black Friday:
- In comparison to last year’s Black Friday Real-Time Crawlers requests volume to one of the most significant marketplaces online, this year’s 2019 Black Friday requests volume grew by a staggering 93%!
Data center proxies requests insights
On the subject of Data center proxies request volume, during the Black Friday, there was a slight request growth noted, as Research Department’s analysis reveals. However, taking into account a substantial Year on Year (YoY) growth in Data center proxies request volume, this was to be expected. Here’s how this year’s Black Friday Data center proxies requests volume compare to historical requests metrics:
- In comparison to the previous 7 days usual request volume, on the Black Friday, Data center proxies request volume grew merely by 1%.
- When comparing to the last 30 days request volume metrics, on the Black Friday, Data center proxies request volume stayed the same.
- However, when comparing the Data center request volume to last year’s Black Friday metrics, a 34% request volume surge was noted.
Furthermore, our Research Department has found out that during Black Friday, there was a significant amount of requests processed to “Clothes” category websites. As we can see from the graph below, a noticeable growth can be observed 10 days before Black Friday i.e., from the 19th of November.
Final Words
Hopefully, you found our Research Department’s insights interesting and useful. Needless to say, but such days as Black Friday are a true goldmine for e-commerce businesses. However, as our carried out analysis shows, switched-on companies heavily rely on the web scraping practice to monitor the competition and optimize their procedures to capitalize on presented opportunities fully.
Also, if you would like to find out how our Research Department supports our valued partners’ web scraping operations, check out our in-depth interview with the data research team.
Lastly, if you have any further questions or would like to get a consultation about your own web scraping project, feel free to drop us a line via live chat or email us at [email protected].
![What Is a Proxy: Web Scraping Basics [2020 Guide]](https://oxylabs.io/blog/images/2020/03/Blog-header-2.png)
