How Websites Block Bots

We have already covered what is a bot and how it works. Now it was time to dig the topic deeper.

On the second day of OxyCon, Dmitry Babitsky, the co-founder & chief scientist at ForNova, went through a few different methods that websites utilize to recognize suspicious behavior, detect bots, and ultimately block them. If you want to dig deeper into what is a bot, then we have covered it in another article.

How websites recognize suspicious behavior? 

Below you’ll find what Mr. D. Babitsky listed as the most popular methods recognizing suspicious behaviour online:

• Large amounts of unusual requests and URL’s.

• Missing cookies – if you don’t have cookies, it is suspicious. However, if you do have cookies – they can track you.

• Miscorrelation between different request attributes – such as the IP address location. Make sure you compare your language and time zone with your IP address’. 

• WebRTC leaking your real IP address. 

• Suspicious browser configuration – e.g., disabled javascript. Different browsers have different javascript. Based on the supported functions on the javascript, the web can double-check you.

• Non-human behavior – if you use the mouse and keyboard, it’ll be fine, but if you use a javascript to click things, it will be easily recognized as a bot. (typing vs. pasting, clicking multiple times on captcha solving, etc.)

• Browser performance analysis and comparison with similar configurations.