avatar

Vytautas Kirjazovas

Dec 05, 2019 6 min read

Less than a week ago, Black Friday madness took over the world, and for many, it was an excellent opportunity to grab irresistible money-saving deals. To paint the picture, Black Friday in the USA took the lead as the busiest day for online shoppers (93.2M), leaving previous historical frontrunner day – Cyber Monday (83.3M) behind. Impressive numbers, isn’t it?

To capture the lion’s share of sales, many companies turn to price tracking and product catalog monitoring practices to unlock a further competitive advantage over their direct rivals. Consequently, businesses timely adjust their pricing tactics and stock, which allows them to win the customer. And, to gather this knowledge, companies increasingly rely on web scraping practice, which is usually supported by a proxy infrastructure.

In this article, we will reveal 2019 Black Friday web scraping insights, supported by actual usage statistics of our clients, that utilized Residential and Datacenter proxies, and our Real-Time Crawler data gathering solution. Also, a special thanks go to our Research Department for providing these valuable insights! Let’s begin. 

Please note: Traffic volumes and request amounts are summarized into trend graphs, ensuring absolute anonymity. The findings don’t include usage statistics of the clients that prefer for their usage statistics to not be recorded.

Residential proxies traffic insights

A data analysis conducted by our Research Department revealed a noteworthy surge in traffic volume via Residential proxies on Black Friday (29th of November), as it can be observed from a noticeable spike in the graph below: 

Residential proxies traffic volume
Residential proxies traffic volume

To put this into perspective, let’s compare this year’s Black Friday Residential proxies traffic volume with historical traffic volume metrics:

  • In comparison to the previous 7 days usual traffic volume, on the Black Friday, Residential proxies traffic volume increased by a significant 20%.
  • When compared to the last 30 days’ usual traffic volume, on the Black Friday, Residential proxies traffic volume increased similarly, by 21%.
  • Whereas, in contrast to 2018 Black Friday’s Residential proxies traffic volume, this year’s Black Friday Residential proxies traffic grew by a substantial 65%.
Residential proxies traffic: Black Friday 2018 vs. Black Friday 2019
Residential proxies traffic: Black Friday 2018 vs. Black Friday 2019

Real-Time Crawler requests volume insights 

A data analysis carried out by our Research Department has identified a surge in the volume of requests processed via our Real-Time Crawler on the Black Friday. To paint the picture, here are the comparisons to historical Real-Time Crawler requests volume vs. 2019 Black Friday requests volume:

  • In comparison to the previous 7 days usual request volume, on the Black Friday Real-Time Crawler noted a 16%request volume growth.
  • When compared to the previous 30 days request volume metrics, on the Black Friday requests volume, in fact, declined by 3%
  • Lastly, in comparison to last year’s Black Friday Real-Time Crawler requests volume, 2019 Black Friday’s request volume grew by a significant 19%
Real-Time Crawler request volume: Black Friday 2018 vs. Black Friday 2019
Real-Time Crawler request volume: Black Friday 2018 vs. Black Friday 2019

As you might be aware, our Real-Time Crawler is a robust web data collection tool, optimized to capture data from search engines and e-commerce websites with 100% data delivery success rates. That said, our Research Department has also revealed a substantial requests growth to one of the most prominent marketplaces via the Real-Time Crawler solution on this year’s Black Friday: 

  • In comparison to last year’s Black Friday Real-Time Crawlers requests volume to one of the most significant marketplaces online, this year’s 2019 Black Friday requests volume grew by a staggering 93%
Real-Time Crawler requests volume (marketplace): Black Friday 2018 vs. Black Friday 2019
Real-Time Crawler requests volume (marketplace): Black Friday 2018 vs. Black Friday 2019

Datacenter proxies requests insights

On the subject of datacenter proxies request volume, during the Black Friday, there was a slight request growth noted, as Research Department’s analysis reveals. However, taking into account a substantial Year on Year (YoY) growth in Datacenter proxies request volume, this was to be expected. Here’s how this year’s Black Friday Datacenter proxies requests volume compare to historical requests metrics:

  • In comparison to the previous 7 days usual request volume, on the Black Friday, Datacenter proxies request volume grew merely by 1%
  • When comparing to the last 30 days request volume metrics, on the Black Friday, Datacenter proxies request volume stayed the same.
  • However, when comparing the Datacenter request volume to last year’s Black Friday metrics, a 34% request volume surge was noted.
Data center proxies volume of requests: Black Friday 2018 vs. Black Friday 2019
Datacenter proxies volume of requests: Black Friday 2018 vs. Black Friday 2019

Furthermore, our Research Department has found out that during Black Friday, there was a significant amount of requests processed to “Clothes” category websites. As we can see from the graph below, a noticeable growth can be observed 10 days before Black Friday i.e., from the 19th of November.

Datacenter proxies volume of requests (Clothes category)
Datacenter proxies volume of requests (Clothes category)

Final words

Hopefully, you found our Research Department’s insights interesting and useful. Needless to say, but such days as Black Friday are a true goldmine for e-commerce businesses. However, as our carried out analysis shows, switched-on companies heavily rely on the web scraping practice to monitor the competition and optimize their procedures to capitalize on presented opportunities fully.

Also, if you would like to find out how our Research Department supports our valued partners’ web scraping operations, check out our in-depth interview with the data research team. 

If you are interested in starting to use our datacenter proxies, register here and scale your business up! Lastly, if you have any further questions or would like to get a consultation about your own web scraping project, feel free to drop us a line via live chat or contact us by clicking here.

avatar

About Vytautas Kirjazovas

Vytautas Kirjazovas is a PR Manager at Oxylabs, and he places a strong personal interest in technology due to its magnifying potential to make everyday business processes easier and more efficient. Vytautas is fascinated by new digital tools and approaches, in particular, for web data harvesting purposes, so feel free to drop him a message if you have any questions on this topic. He appreciates a tasty meal, enjoys travelling and writing about himself in the third person.

Related articles

Choosing Between Residential and Datacenter Proxies

Choosing Between Residential and Datacenter Proxies

Jun 19, 2020

3 min read

All information on Oxylabs Blog is provided on an "as is" basis and for informational purposes only. We make no representation and disclaim all liability with respect to your use of any information contained on Oxylabs Blog or any third-party websites that may be linked therein. Before engaging in scraping activities of any kind you should consult your legal advisors and carefully read the particular website's terms of service or receive a scraping license.