avatar

Gabija Fatenaite

Oct 02, 2019 4 min read

New day, new horizons. After the first day concluded with a workshop, dinner & some entertainment, OxyCon participants arrived for the second part of the conference, ready for more action. 

OxyCon continued with some amazing mini croissants and three presentations, which were so good that the deliciousness of the French pastries paled in comparison. Also, if you haven’t read it yet, we kindly invite you to check out the recap of the first day.

OxyCon 2019: The Top Takeaways From Day Two #2

Oops I Scraped. Should I Hire a Lawyer? 

While giving quite a few tongue in cheek legal disclaimers throughout their presentation, Oxylabs in-house law guys Denas Grybauskas and Nerijus Sveistys discussed some legal cases. Here are the lawsuits in question, relevant to how copyright and web scraping are interpreted today from a legal standpoint:

  • Feist Publications v. Rural Telephone Service Co. (1991)
  • Ebay v. Bidder’s Edge (2000)
  • Power Ventures v. Facebook (2009)
  • Craigslist v. 3taps Inc. (2013)
  • QVC v. Resultly (2014)
  • Ryanair v. PR Aviation (2015)
  • Ryanair v. Expedia (2019)
  • HiQ labs v. LinkedIn (2019)

One of the key takeaways from their presentation was that although the famous HiQ labs v. LinkedIn court decision was beneficial for companies engaging in web scraping, there is still a lot of uncertainty legally. A handy scheme was also provided in regards to the level of risk related to various different scraping approaches.

OxyCon 2019: The Top Takeaways From Day Two #2

How Websites Block Bots

Dmitry Babitsky, co-founder & chief scientist @ ForNova detailed a bunch of different methods that websites utilize to recognize, track and ultimately block bots. Here are some of the most popular methods:

  1. Big amount of unusual requests and URLs. 
  2. Missing cookies.
  3. Miscorrelation between different request attributes, such as the IP address location not matching to browser languages and the time zone.
  4. WebRTC leaking your real IP address. 
  5. Suspicious browser configuration, e.g. disabled Javascript.
  6. Browser performance analysis and comparison with similar configurations.
  7. Analyzing mouse and keyboard inputs.

Mr. Babitsky also touched upon more sophisticated methods of identification, such as browser fingerprinting, which our speaker Allen O’Neill discussed in-depth yesterday, and put forward some of the things that websites do after identifying a bot.  

OxyCon 2019: The Top Takeaways From Day Two #3

Q&A: Reaching a 100% Success Rate

Oxylabs Software Engineer Eivydas Vilcinskas did a great presentation on how to reach a 100% scraping success rate using Oxylabs Real-Time Crawler, our advanced scraping solution that does all of the heavy lifting for you. It was a walk through all the challenges one might encounter while using Real-Time Crawler with some tips on how to solve each and every one. 

However, if you use Real-Time Crawler and could not make it to the conference, we have good news – a detailed documentation with all of the same information is currently in the works and you know you can always rely on our premium support for whatever you need.

Read the detailed recap of the presentation.

After an additional Q&A that got even more in-depth, with many attendees curious about the underpinnings of our Real-Time Crawler, the one and only – Head of Account Management Mante Petrauskaite said some closing remarks on how OxyCon came to life, thanked all of the speakers, attendees and the Oxy folks who made the event possible. 

OxyCon 2019: The Top Takeaways From Day Two #4

Truly, OxyCon was an inspiring conference with some of the smartest people from every corner of the world gathering in one place and we would once again like to thank everyone who attended. See you next year!

P.S. Just to stay updated, be sure to follow Oxylabs on LinkedIn and Twitter.

avatar

About Gabija Fatenaite

Gabija Fatenaite is a Content Manager at Oxylabs. Having grown up on video games and the internet, she grew to find the tech side of things more and more interesting over the years. So if you ever find yourself wanting to learn more about proxies (or video games), feel free to contact her - she’ll be more than happy to answer you.

Related articles

Using Web Scraping for Lead Generation

Using Web Scraping for Lead Generation

Nov 06, 2019

4 min read

Scraping the Web With 100% Success Rate

Scraping the Web With 100% Success Rate

Oct 10, 2019

6 min read

Scraping Trends and Infrastructure Sustainability

Scraping Trends and Infrastructure Sustainability