While web scraping and automation tools have developed significantly during recent years, handling dynamic websites has become a breeze. Headless browsers without a graphical user interface offered an efficient way of collecting public data, as you can control them programmatically. If combined with proxies, they’re even better. In this article, we’ll go through the Puppeteer integration process with Oxylabs Residential Proxies.
Puppeteer is a Node.js library offering a top-level API for headless Chrome or Chromium browsers’ control through the DevTools Protocol. It’s also known as a Headless Chrome Node API and is used for automating Chrome to run website tests.
Before getting started with Puppeteer, you’ll need to install some basic tools: Node.js and a code editor of your choice. After that, create a Node.js project and install the required packages. You may find a detailed guide on how to install and run Puppeteer on our blog post.
Once everything is set up, we can move on to the next part – Oxylabs Residential Proxies integration with Puppeteer.
Within Puppeteer, fill in the value, for example:
pr.oxylabs.io:7777
Under ‘page.authenticate’ input your Oxylabs proxy sub-user username in the ‘username’ value and your password.
The example of a code looks like this:
const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch({
headless: false,
args: ['--proxy-server=pr.oxylabs.io:7777]
});
const page = await browser.newPage();
await page.authenticate({
username: 'USERNAME',
password: 'PASSWORD'
});
await page.goto('https://ip.oxylabs.io');
await page.screenshot({path: 'example.png'});
await browser.close();
})();
You can also use country-specific entries. For example, if you put us-pr.oxylabs.io under ‘host’ and 10001 under ‘port’, you will receive a US exit node with a sticky session. Please check out our documentation for a complete list of country-specific entry nodes.
And that’s it! You’ve successfully integrated Oxylabs Residential proxies with Puppeteer.
Puppeteer, in combination with Oxylabs Residential Proxies, might be of great help when it comes to scraping public data from dynamic websites.
In case you have any queries about integrating Oxylabs proxies, please get in touch with us at any time.
About the author
Monika Maslauskaite
Former Content Manager
Monika Maslauskaite is a former Content Manager at Oxylabs. A combination of tech-world and content creation is the thing she is super passionate about in her professional path. While free of work, you’ll find her watching mystery, psychological (basically, all kinds of mind-blowing) movies, dancing, or just making up choreographies in her head.
All information on Oxylabs Blog is provided on an "as is" basis and for informational purposes only. We make no representation and disclaim all liability with respect to your use of any information contained on Oxylabs Blog or any third-party websites that may be linked therein. Before engaging in scraping activities of any kind you should consult your legal advisors and carefully read the particular website's terms of service or receive a scraping license.
Integrations
Learn how to integrate Apify with proxies to ensure a smooth scraping process using Learn how to integrate Oxylabs’ Datacenter and Residential Proxies in this step-by-step guide.
Roberta Aukstikalnyte
2022-04-05
Integrations
Use proxies with the Aezakmi anti-detection browser to enhance anonymity. Learn how to integrate Oxylabs’ Datacenter and Residential Proxies in this step-by-step guide.
Monika Maslauskaite
2021-11-16
Integrations
Extracting data might sometimes be troublesome. To make this process easier, learn how to integrate Oxylabs Residential Proxies with ParseHub tool.
Jolita Pundzaite
2021-11-05
Get the latest news from data gathering world
Scale up your business with Oxylabs®
GET IN TOUCH
General:
hello@oxylabs.ioSupport:
support@oxylabs.ioCareer:
career@oxylabs.ioCertified data centers and upstream providers
Connect with us
Resources
Innovation hub