Active Fingerprinting With Pyppeteer and Playwright to Avoid Bot Detection

Web scraping for developers, live presentation
26 August, 12:40PM BST, 50min

Most people have heard about website fingerprinting as the practice of websites tracking internet users and pinpointing their identity even if the users never asked for it. However, the fingerprinting techniques used for user tracking can also be used for detecting headless browsers, which impedes scraping JS-reliant websites.

In this talk, we will take a quick look at fingerprinting and how it works. Paulius Stundžia, Oxylabs Software Engineer, will demonstrate how websites can use fingerprinting techniques to detect headless browsers and what can be done to circumvent that.

Paulius will also perform a live coding session with two of the most popular Python libraries for headless browsing: pyppeteer and playwright.

To those who would like to follow along, Paulius suggests some prep work:

Make sure you have Python 3. Preferably, Python 3.8+
Install pyppeteer and playwright in your virtual environment

Read About OxyCon Day Two

Please note: The views expressed by speakers or moderators are those of the speaker or moderators and not, necessarily, of Oxylabs or other respective organizations. Before engaging in scraping activities of any kind, you should consult your legal advisors.

Keep up with the future of web scraping

Read About OxyCon

Meet the speaker

Paulius Stundžia, Software Engineer at Oxylabs

Paulius has been a software engineer with Oxylabs for over three years now and is a former OxyCon (2019) speaker.

He started with Oxylabs as a Python developer and took the lead on implementing headless scraping in one of Oxylabs most powerful web scraping tools. He has worked on services in Oxylabs scraping infrastructure in both Python and PHP. Paulius has recently moved to Oxylabs residential proxies team as a Go developer, where he still maintains many Python-based services.