Oxylabs Announces Partnership With Student-led TrackCorona Website

Learn more
avatar

Vytautas Kirjazovas

Dec 05, 2019 5 min read

Less than a week ago, Black Friday madness took over the world, and for many, it was an excellent opportunity to grab irresistible money-saving deals. To paint the picture, Black Friday in the USA took the lead as the busiest day for online shoppers (93.2M), leaving previous historical frontrunner day – Cyber Monday (83.3M) behind. Impressive numbers, isn’t it?

To capture the lion’s share of sales, many companies turn to price tracking and product catalog monitoring practices to unlock a further competitive advantage over their direct rivals. Consequently, businesses timely adjust their pricing tactics and stock, which allows them to win the customer. And, to gather this knowledge, companies increasingly rely on web scraping practice, which is usually supported by a proxy infrastructure.

In this article, we will reveal 2019 Black Friday web scraping insights, supported by actual usage statistics of our clients, that utilized Residential and Data center proxies, and our Real-Time Crawler data gathering solution. Also, a special thanks go to our Research Department for providing these valuable insights! Let’s begin. 

Please note: Traffic volumes and request amounts are summarized into trend graphs, ensuring absolute anonymity. The findings don’t include usage statistics of the clients that prefer for their usage statistics to not be recorded.

Residential proxies traffic insights

A data analysis conducted by our Research Department revealed a noteworthy surge in traffic volume via Residential proxies on Black Friday (29th of November), as it can be observed from a noticeable spike in the graph below: 

Residential proxies traffic volume
Residential proxies traffic volume

To put this into perspective, let’s compare this year’s Black Friday Residential proxies traffic volume with historical traffic volume metrics:

  • In comparison to the previous 7 days usual traffic volume, on the Black Friday, Residential proxies traffic volume increased by a significant 20%.
  • When compared to the last 30 days’ usual traffic volume, on the Black Friday, Residential proxies traffic volume increased similarly, by 21%.
  • Whereas, in contrast to 2018 Black Friday’s Residential proxies traffic volume, this year’s Black Friday Residential proxies traffic grew by a substantial 65%.
Residential proxies traffic: Black Friday 2018 vs. Black Friday 2019
Residential proxies traffic: Black Friday 2018 vs. Black Friday 2019

Real-Time Crawler requests volume insights 

A data analysis carried out by our Research Department has identified a surge in the volume of requests processed via our Real-Time Crawler on the Black Friday. To paint the picture, here are the comparisons to historical Real-Time Crawler requests volume vs. 2019 Black Friday requests volume:

  • In comparison to the previous 7 days usual request volume, on the Black Friday Real-Time Crawler noted a 16%request volume growth.
  • When compared to the previous 30 days request volume metrics, on the Black Friday requests volume, in fact, declined by 3%
  • Lastly, in comparison to last year’s Black Friday Real-Time Crawler requests volume, 2019 Black Friday’s request volume grew by a significant 19%
Real-Time Crawler request volume: Black Friday 2018 vs. Black Friday 2019
Real-Time Crawler request volume: Black Friday 2018 vs. Black Friday 2019

As you might be aware, our Real-Time Crawler is a robust web data collection tool, optimized to capture data from search engines and e-commerce websites with 100% data delivery success rates. That said, our Research Department has also revealed a substantial requests growth to one of the most prominent marketplaces via the Real-Time Crawler solution on this year’s Black Friday: 

  • In comparison to last year’s Black Friday Real-Time Crawlers requests volume to one of the most significant marketplaces online, this year’s 2019 Black Friday requests volume grew by a staggering 93%
Real-Time Crawler requests volume (marketplace): Black Friday 2018 vs. Black Friday 2019
Real-Time Crawler requests volume (marketplace): Black Friday 2018 vs. Black Friday 2019

Data center proxies requests insights

On the subject of Data center proxies request volume, during the Black Friday, there was a slight request growth noted, as Research Department’s analysis reveals. However, taking into account a substantial Year on Year (YoY) growth in Data center proxies request volume, this was to be expected. Here’s how this year’s Black Friday Data center proxies requests volume compare to historical requests metrics:

  • In comparison to the previous 7 days usual request volume, on the Black Friday, Data center proxies request volume grew merely by 1%
  • When comparing to the last 30 days request volume metrics, on the Black Friday, Data center proxies request volume stayed the same.
  • However, when comparing the Data center request volume to last year’s Black Friday metrics, a 34% request volume surge was noted.
Data center proxies volume of requests: Black Friday 2018 vs. Black Friday 2019
Data center proxies volume of requests: Black Friday 2018 vs. Black Friday 2019

Furthermore, our Research Department has found out that during Black Friday, there was a significant amount of requests processed to “Clothes” category websites. As we can see from the graph below, a noticeable growth can be observed 10 days before Black Friday i.e., from the 19th of November.

Data center proxies volume of requests (Clothes category)
Data center proxies volume of requests (Clothes category)

Final Words

Hopefully, you found our Research Department’s insights interesting and useful. Needless to say, but such days as Black Friday are a true goldmine for e-commerce businesses. However, as our carried out analysis shows, switched-on companies heavily rely on the web scraping practice to monitor the competition and optimize their procedures to capitalize on presented opportunities fully.

Also, if you would like to find out how our Research Department supports our valued partners’ web scraping operations, check out our in-depth interview with the data research team. 

Lastly, if you have any further questions or would like to get a consultation about your own web scraping project, feel free to drop us a line via live chat or email us at [email protected].

avatar

About Vytautas Kirjazovas

Vytautas Kirjazovas is a PR Manager at Oxylabs, and he places a strong personal interest in technology due to its magnifying potential to make everyday business processes easier and more efficient. Vytautas is fascinated by new digital tools and approaches, in particular, for web data harvesting purposes, so feel free to drop him a message if you have any questions on this topic. He appreciates a tasty meal, enjoys travelling and writing about himself in the third person.

Related articles

Proxy vs VPN

Proxy vs VPN

Mar 18, 2020

8 min read

What Is a Proxy: Web Scraping Basics [2020 Guide]

What Is a Proxy: Web Scraping Basics [2020 Guide]

Mar 17, 2020

25 min read

Gathering Data for Sales Intelligence

Gathering Data for Sales Intelligence

Mar 10, 2020

6 min read

All information on Oxylabs Blog is provided on an “as is” basis and for informational purposes only. We make no representation with respect to the accuracy, completeness, availability or otherwise of any information contained on Oxylabs Blog / any third-party websites that may be linked therein. We disclaim all liability for any errors, omissions or delays in such information and for any losses, injuries or damages that may arise out of its use.