Oxylabs Residential Proxies With Puppeteer
avatar

Monika Maslauskaite

Aug 19, 2021 2 min read

While web scraping and automation tools have developed significantly during recent years, handling dynamic websites has become a breeze. Headless browsers without a graphical user interface offered an efficient way of collecting public data, as you can control them programmatically. If combined with proxies, they’re even better. In this article, we’ll go through the Puppeteer integration process with Oxylabs Residential Proxies.

What is Puppeteer?

Puppeteer is a Node.js library offering a top-level API for headless Chrome or Chromium browsers’ control through the DevTools Protocol. It’s also known as a Headless Chrome Node API and is used for automating Chrome to run website tests. 

How to integrate Oxylabs proxies with Puppeteer? 

Before getting started with Puppeteer, you’ll need to install some basic tools: Node.js and a code editor of your choice. After that, create a Node.js project and install the required packages. You may find a detailed guide on how to install and run Puppeteer on our blog post

Once everything is set up, we can move on to the next part – Oxylabs Residential Proxies integration with Puppeteer. 

  • Within Puppeteer, fill in the value, for example:
pr.oxylabs.io:7777
  • Under ‘page.authenticate’ input your Oxylabs proxy sub-user username in the ‘username’ value and your password.
  • The example of a code looks like this:
const puppeteer = require('puppeteer');
(async () => {
  const browser = await puppeteer.launch({
    headless: false,
    args: ['--proxy-server=pr.oxylabs.io:7777]  
});
  const page = await browser.newPage();
    await page.authenticate({
        username: 'USERNAME',
        password: 'PASSWORD'
    });
    await page.goto('https://ip.oxylabs.io');
    await page.screenshot({path: 'example.png'});
    await browser.close();
})();

You can also use country-specific entries. For example, if you put us-pr.oxylabs.io under ‘host’ and 10001 under ‘port’, you will receive a US exit node with a sticky session. Please check out our documentation for a complete list of country-specific entry nodes. 

And that’s it! You’ve successfully integrated Oxylabs Residential proxies with Puppeteer. 

Conclusion

Puppeteer, in combination with Oxylabs Residential Proxies, might be of great help when it comes to scraping public data from dynamic websites.

In case you have any queries about integrating Oxylabs proxies, please get in touch with us at any time.

avatar

About Monika Maslauskaite

Monika Maslauskaite is a Content Manager at Oxylabs. A combination of tech-world and content creation is the thing she is super passionate about in her professional path. While free of work, you’ll find her watching mystery, psychological (basically, all kinds of mind-blowing) movies, dancing, or just making up choreographies in her head.

All information on Oxylabs Blog is provided on an "as is" basis and for informational purposes only. We make no representation and disclaim all liability with respect to your use of any information contained on Oxylabs Blog or any third-party websites that may be linked therein. Before engaging in scraping activities of any kind you should consult your legal advisors and carefully read the particular website's terms of service or receive a scraping license.

Related articles

Oxylabs Proxy Integration With Undetectable.io

Oxylabs Proxy Integration With Undetectable.io

Aug 11, 2021

3 min read