Web Unblocker leverages ML-driven proxy management, dynamic browser fingerprinting, ML-powered response recognition, and other advanced technologies to unlock even the most difficult sites.
Follow this guide to learn how to set up Web Unblocker hassle-free.
Setting up Web Unblocker
To start using Web Unblocker, create an account in the Oxylabs dashboard and select your preferred plan. If you want to give Web Unblocker a test run before buying, we offer a free trial accessible after logging into the dashboard.
Afterward, a setup window will appear, prompting you to create an API user.
Creating an API user
To create your API user, simply come up with a username and password and hit Create API user.
Making a basic cURL request
Once you’re done creating your first API user, you’ll receive a cURL example. You can paste it into your terminal or any other setup you prefer and test how it works.
Note that the cURL sample already has some pre-set parameters like location x-oxylabs-geo-location: United States and JavaScript rendering x-oxylabs-render: html. The latter can be implemented when experiencing low success rates or retrieving empty content.
See the image below for a breakdown of cURL structure:
For your convenience, we’ve also added the cURL example in code format:
curl 'http://quotes.toscrape.com/js/'\
-U 'USERNAME:PASSWORD' -x 'unblock.oxylabs.io:60000'\
-H 'x-oxylabs-geo-location: United States'\
-H 'x-oxylabs-render: html' -k -v Once you’ve sent the request, you should get an output similar to the one below:
HTTP/1.1 200 OK
X-Job-Id: 1234567890123456
X-Session-Id: 123randomString
cf-cache-status: DYNAMIC
cf-ray: 55c2ab837eddcba8-VIE
content-encoding: gzip
content-length: 72657
content-type: text/html
date: Tue, 1 Jan 2020 00:00:01 GMT,expect-ct: max-age=604800,report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
last-modified: Tue, 1 Dec 2020 00:00:00 GMT
server: cloudflare
status: 200
strict-transport-security: max-age=31536000
X-DNS-Prefetch-Control: off
Set-Cookie: NID=1234567890; expires=Wed, 29-Jul-2020 10:56:21 GMT
Set-Cookie: 1P_JAR=0987654321; expires=Wed, 29-Jul-2020 10:56:21 GMT
x-cache: MISS from localhost
x-cache-lookup: MISS from localhost:3129
<html>content here</html>Check out the video below for a visual guide:
Additional functionalities
Web Unblocker employs pre-defined cookies, headers, and sessions to optimize website unblocking. So, it’s best that you refrain from sending any custom parameters commonly used for unblocking, as they may interfere with Web Unblocker’s ability to get quality data.
That being said, for special cases, we support a list of additional functionalities for target-specific requests:
| Parameter | Description |
|---|---|
X-Oxylabs-Session-Id |
To reuse the same IP for multiple requests, add a session ID, which can be a random string of characters. |
X-Oxylabs-Geo-Location |
To use an IP address from a specific location. |
| Headers | To send custom headers, use x-oxylabs-force-headers: 1 when submitting your request. |
| Cookies | To add custom cookies to your requests. |
X-Oxylabs-Status-Code |
If your target site returns a non-standard status code with a successful response, you can send the response status code, and our system will not retry the request. |
X-Oxylabs-Render |
To render JavaScript, use html to get a rendered HTML or png to get a screenshot of the page. You can also define your own browser instructions, such as clicking, inputting, and more. List of instructions. |
POST requests |
Web Unblocker allows you to send a POST request to a web endpoint of your choice. You can send data to a target website, making it return a different result. |
Response codes
See the following table for a list of some of the most common status codes indicating errors:
| Code | Status | Description |
|---|---|---|
200 |
OK | All went well. |
400 |
Multiple error messages | Wrong request structure. Perhaps a misspelled parameter or an invalid value. Find more information in the response body. |
401 |
Authorization header not provided/Invalid authorization header/Client not found | Missing authorization header or incorrect login credentials. |
403 |
Forbidden | This resource isn't available to your account. |
404 |
Not Found | The job ID you're looking for is no longer available. |
408 |
Timeout | The request timed out. |
429 |
Too many requests | You've exceeded the rate limit. |
500, 502, 503 |
Internal Server Error | We're experiencing some issues. We're probably aware of the issues, but you can report it anyway. |
550 |
Faulted After Too Many Retries | Something went wrong. Try again at no extra cost or get in touch with us. |
If you run into any other response codes, contact our Support Team for assistance.
Usage statistics
In the dashboard, you will find a Statistics tab which enables you to track your traffic usage. Here, you’ll find a detailed view of how much traffic you use daily. You can set the time period of usage as preferred and filter the usage by sub-user and domain.
Additionally, you’ll be able to see your requests’ success rates, average response times, and the number of results for each domain.
Additional resources
If you have any further questions, please don't hesitate to get in touch with our Support Team.
Also, feel free to explore other developer-focused resources for more insights:
People also ask
How is my traffic usage calculated when using Web Unblocker?
To ensure that you get the most value out of Web Unblocker, we only charge you for successfully extracted data. This means that failed scraping attempts won’t be included in your traffic usage. Moreover, we calculate both upstream and downstream traffic. For more information on billing, check out our documentation.
How does the auto-retry feature work?
To ensure the highest success rates, we implement the auto-retry feature. This means that when a scraping task fails, our system chooses a new set of parameters and resends the request.
Does Web Unblocker utilize a different proxy pool?
Web Unblocker leverages Oxylabs proxy infrastructure, including 102M+ ethically gathered IPs. By combining Residential and Datacenter Proxy pools, Web Unblocker ensures you can effortlessly unlock data in over 195 countries.
Can I get a Web Unblocker free trial?
Yes, we offer a free trial for our Web Unblocker solution. The trial lasts for one week and you can use up to 1GB (up to 10k results). To claim your free trial, simply register or log in to the dashboard.
What are the pricing options for Web Unblocker?
No matter if you’re a small business or a large enterprise, we offer a wide variety of payment options. Depending on your data needs, you can choose from Regular and Enterprise tiers, each offering four distinct subscription plans:
Regular:
1-week Free trial (1GB or 10K results) — $0
Micro (5GB) — $75/mo
Starter (25GB) — $325/mo
Advanced (60GB) — $660/mo
Enterprise:
Venture (100GB) — $900/mo
Business (250GB) — $2000/mo
Corporate (500GB) — $3500/mo
Custom+ (1TB+) — $5000/mo
For more information about our pricing, check out our pricing page.
About the author
Vytenis Kaubre
Copywriter
Vytenis Kaubre is a Copywriter at Oxylabs. As his passion lay in creative writing and curiosity in anything tech kept growing, he joined the army of copywriters. After work, you might find Vytenis watching TV shows, brainstorming ideas for tabletop games, or taking Raymond Chandler’s advice, “When in doubt, have a man come through a door with a gun in his hand” too seriously (when writing short stories).
All information on Oxylabs Blog is provided on an "as is" basis and for informational purposes only. We make no representation and disclaim all liability with respect to your use of any information contained on Oxylabs Blog or any third-party websites that may be linked therein. Before engaging in scraping activities of any kind you should consult your legal advisors and carefully read the particular website's terms of service or receive a scraping license.
Related articles
Get the latest news from data gathering world
Web Unblocker for the most complex targets
Scrape public data from almost any target without getting blocked.
Scale up your business with Oxylabs®
GET IN TOUCH
General:
hello@oxylabs.ioSupport:
support@oxylabs.ioCareer:
career@oxylabs.io
Certified data centers and upstream providers
Connect with us
Advanced proxy solutions
Resources
Innovation hub