Web Unblocker Quick Start Guide

Vytenis Kaubrė

Last updated by Dovydas Vėsa

2023-04-06

3 min read

AI Summary:

Learn how to quickly set up and start using Web Unblocker to access public data from complex websites. This guide walks through the essential steps, from creating an API user and sending your first request to understanding key parameters and response codes. It also covers additional features such as geo-targeting, session control, and JavaScript rendering to help you get the most out of Web Unblocker.

Web Unblocker leverages ML-driven proxy management, dynamic browser fingerprinting, ML-powered response recognition, and other advanced technologies to collect data from even the most complex sites.

Follow this guide to learn how to set up Web Unblocker hassle-free. In addition, you can teach your coding assistant or AI agent all there is to know about the Web Unblocker in just a few steps with Agent Skills. Learn how.

Setting up Web Unblocker

1. Register and create an instance

Start by heading to the Oxylabs Dashboard and create an account. If this is your first time with us, you’ll receive a $5 welcome credit to try Web Unblocker for free.

Then, click + Add product instance and select Web Unblocker in the Proxy Solutions category. Finally, choose the instance name, if required – set product usage limits, and create the instance.

Add Oxylabs product instance in the dashboard

2. Get Web Unblocker credentials

Open your Web Unblocker product instance from the dashboard home section. There, look for your new instance username and password at the top of the overview. These credentials are your access to the Web Unblocker.

Find Oxylabs product instance credentials

Making a basic cURL request

In the Overview of your Web Unblocker instance you'll find a collection of pre-made request examples, including cURL. You can paste and run it in your terminal or any other setup you prefer to test how it works.

Note that the cURL sample is basic and only uses the gateway, you credentials, and the target. If you require additional functions, such as JavaScript rendering of a web page, you can add additional headers for your query (e.g. x-oxylabs-render: html).

See the image below for a breakdown of cURL structure:

A basic test query with additional header

For your convenience, we’ve also added the cURL example in code format:

curl -k -x unblock.oxylabs.io:60000 -U 'USERNAME:PASSWORD' 'https://sandbox.oxylabs.io/products/' -H 'x-oxylabs-geo-location: United States'

Once you’ve sent the request, you should get a similar output with the complete DOM as HTML, including JavaScript-rendered content:

<!DOCTYPE html><html>fully rendered content here</html>

Additional functionalities

Web Unblocker employs pre-defined cookies, headers, and sessions to optimize reliable access to the website. So, it’s best that you refrain from sending any custom parameters commonly used for accessing, as they may interfere with Web Unblocker’s ability to get quality data.

That being said, for special cases, we support a list of additional functionalities for target-specific requests:

Parameter	Description
`X-Oxylabs-Session-Id`	To reuse the same IP for multiple requests, add a session ID, which can be a random string of characters.
`X-Oxylabs-Geo-Location`	To use an IP address from a specific location.
Headers	To send custom headers, use `x-oxylabs-force-headers: 1` when submitting your request.
Cookies	To add custom cookies to your requests.
`X-Oxylabs-Status-Code`	If your target site returns a non-standard status code with a successful response, you can send the response status code, and our system will not retry the request.
`X-Oxylabs-Render`	To render JavaScript, use `html` to get a rendered HTML or `png` to get a screenshot of the page. You can also define your own browser instructions, such as clicking, inputting, and more. List of instructions.
`POST` requests	Web Unblocker allows you to send a POST request to a web endpoint of your choice. You can send data to a target website, making it return a different result.

Response codes

See the following table for a list of some of the most common status codes indicating errors:

Code	Status	Description
`200`	OK	All went well.
`400`	Multiple error messages	Wrong request structure. Perhaps a misspelled parameter or an invalid value. Find more information in the response body.
`401`	Authorization header not provided/Invalid authorization header/Client not found	Missing authorization header or incorrect login credentials.
`403`	Forbidden	This resource isn't available to your account.
`404`	Not Found	The job ID you're looking for is no longer available.
`408`	Timeout	The request timed out.
`429`	Too many requests	You've exceeded the rate limit.
`500`, `502`, `503`	Internal Server Error	We're experiencing some issues. We're probably aware of the issues, but you can report it anyway.
`550`	Faulted After Too Many Retries	Something went wrong. Try again at no extra cost or get in touch with us.

If you run into any other response codes, contact our Support Team for assistance.

Usage statistics

The Usage section of your product instance lets you see detailed information on your Web Unblocker usage using numerous filters. You can see usage graphs for any selected period and filter by targets or countries and a table of your daily usage breakdowns.

Additionally, you’ll be able to see your requests’ success rates, average response times, and the number of results for each domain.

Web Unblocker usage daily breakdown table

Additional resources

If you have any further questions, please don't hesitate to get in touch with our Support Team.

Also, feel free to explore other developer-focused resources for more insights:

People also ask

How is my traffic usage calculated when using Web Unblocker?

To ensure that you get the most value out of Web Unblocker, we only charge you for successfully extracted data. This means that failed scraping attempts won’t be included in your traffic usage. Moreover, we calculate both upstream and downstream traffic. For more information on billing, check out our documentation.

How does the auto-retry feature work?

Does Web Unblocker utilize a different proxy pool?

Can I get a Web Unblocker free trial?

What are the pricing options for Web Unblocker?

We offer different monthly subscription plans to suit individual needs. Regular plans found in self-service operate on a credit-based system where one subscription gives access to all Oxylabs products and your credit is billed per product usage. For Agent Browser you can choose from:

Free – start with $5 welcome credit ($7.2/GB, up to 0.7 GB)
Starter – $30/mo ($7.2/GB, up to 4 GB)
Basic – $100/mo ($6/GB, up to 16 GB)
Flexible – $500/mo ($4.8/GB, up to 104 GB)

Custom Enterprise plans are available via the sales team and start at $2500/month ($3/GB, 833+ GB).

For more information about our pricing, check out our pricing page. If you need any help with choosing a plan or getting started with a product, make sure to contact our expert team.

About the author

Vytenis Kaubrė

Technical Content Researcher

Vytenis Kaubrė was a Technical Content Researcher at Oxylabs. Creative writing and a growing interest in technology fueled his work, where he researched and crafted technical content while honing his skills in Python. Off duty, he could often be found working on personal projects, learning about cybersecurity, or relaxing with a book.

Learn more about the author Vytenis Kaubrė Learn more about the author Vytenis Kaubrė

All information on Oxylabs Blog is provided on an "as is" basis and for informational purposes only. We make no representation and disclaim all liability with respect to your use of any information contained on Oxylabs Blog or any third-party websites that may be linked therein. Before engaging in scraping activities of any kind you should consult your legal advisors and carefully read the particular website's terms of service or receive a scraping license.