How to Set Up a Proxy Server on Screaming Frog

Building a solid SEO strategy requires fresh and accurate search engine results data. One of the ways to get this data in bulk is by using web scraping tools and proxies. In fact, proxies are an inseparable part of web scraping, helping you bypass certain anti-bot measures. Scraping public SEO data is no exception – this time, we’ll demonstrate how to set up Oxylabs’ Residential and Datacenter Proxies with the Screaming Frog SEO Spider tool. 

How to Set Up a Proxy Server on Screaming Frog

What is Screaming Frog? 

Screaming Frog is a platform designed to scrape public web data for SEO-related purposes (specifics below), available on macOS, Linux, and Windows devices. The tool offers public data scraping from up to 500 URLs; if you need more, you’ll need to upgrade to a bigger plan. 

Here are some common SEO-related initiatives professionals use this tool for: 

Screaming Frog SEO Spider proxy integration 

For this tutorial, we’ll be using macOS 12.6.1 

Before we get started with the integration, download and install the program. If you have the paid version, navigate to the menu bar above and click Enter License. 

A window will pop up, requiring you to enter your username and license key. Enter those and click OK and Restart. You’ll have received these details in your email upon registration. 

1. Locate the Configuration menu > select System > select Proxy.

2. Insert necessary proxy details.

A new window will pop up where you’ll have to enter proxy details. Make sure to tick the Use Proxy Server box. 

2.1. Residential Proxies 

If you’re using Oxylabs’ Residential Proxies, enter pr.oxylabs.io in the Address field and 7777 in the Port field. Then, click OK and Restart to complete the connection:

You can also connect to a proxy in a specific country of your choice. For example, if you enter us-pr.oxylabs.io in the Address field and 10000 under Port, you’ll connect to a US-based proxy server. There are plenty of other countries you can connect to, including sticky session options – check out our documentation for more information on that. 

2.2. Dedicated Datacenter Proxies

The process of connecting to Dedicated Datacenter Proxies is fairly similar – you’ll just need to fill in different details. In the Address field, you’ll need to enter a specific numerical IP address from the acquired list (refer to our documentation for more information.) We’ll use 1.2.3.4 for the sake of this tutorial. In the port field, please enter 60000.

2.3. Shared Datacenter Proxies

For Shared Datacenter Proxies, you’ll need to enter dc.pr.oxylabs.io in the Address field and 10000 under Port, then, click OK and Restart. 

Once again, you can use a country-specific entry. If you enter dc.de-pr.oxylabs.io under Address and 40000 under Port, you’ll connect to a Germany-based proxy server. To see the full list of country-specific entry nodes, head to our documentation. 

4. Check your IP

To see if your connection was successful, head to the What Is My IP Address website and see if your original IP address was changed to that of a proxy.

That’s it! You can now start scraping public SEO data without worrying about blocks.

Wrapping up  

Combined with Oxylabs proxies, Screaming Frog can deliver public search engine data in real time while bypassing anti-bot measures at the same time. 

As the tutorial shows, setting up proxies on this platform is fairly simple, so you’ll be able to start working on your SEO projects right away. If you have questions about integrating Oxylabs products, don’t hesitate to drop us a message at support@oxylabs.io or reach out via live chat. 

Please be aware that this is a third-party tool not owned or controlled by Oxylabs. Each third-party provider is responsible for its own software and services. Consequently, Oxylabs will have no liability or responsibility to you regarding those services. Please carefully review the third party's policies and practices and/or conduct due diligence before accessing or using third-party services.

Get the latest news from data gathering world

I'm interested

Get Screaming Frog proxies for $15/GB