Web Unblocker leverages ML-driven proxy management, dynamic browser fingerprinting, ML-powered response recognition, and other advanced technologies to unlock even the most difficult sites.
Follow this guide to learn how to set up Web Unblocker hassle-free.
To start using Web Unblocker, create an account in the Oxylabs dashboard and select your preferred plan. If you want to give Web Unblocker a test run before buying, we offer a free trial accessible after logging into the dashboard.
Afterward, a setup window will appear, prompting you to create an API user.
To create your API user, simply come up with a username and password and hit Create API user.
Once you’re done creating your first API user, you’ll receive a cURL example. You can paste and run it in your terminal or any other setup you prefer to test how it works.
Note that the cURL sample already has some pre-set parameters like location x-oxylabs-geo-location: United States. If you require JavaScript rendering of a web page, add x-oxylabs-render: html.
See the image below for a breakdown of cURL structure:
A test query from the dashboard
For your convenience, we’ve also added the cURL example in code format:
curl 'https://sandbox.oxylabs.io/products/' -U 'USERNAME:PASSWORD' -x 'unblock.oxylabs.io:60000' -H 'x-oxylabs-geo-location: United States' -k
Once you’ve sent the request, you should get an output similar to the one below:
HTTP/1.1 200 OK
X-Job-Id: 1234567890123456
X-Session-Id: 123randomString
cf-cache-status: DYNAMIC
cf-ray: 55c2ab837eddcba8-VIE
content-encoding: gzip
content-length: 72657
content-type: text/html
date: Tue, 1 Jan 2020 00:00:01 GMT,expect-ct: max-age=604800,report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
last-modified: Tue, 1 Dec 2020 00:00:00 GMT
server: cloudflare
status: 200
strict-transport-security: max-age=31536000
X-DNS-Prefetch-Control: off
Set-Cookie: NID=1234567890; expires=Wed, 29-Jul-2020 10:56:21 GMT
Set-Cookie: 1P_JAR=0987654321; expires=Wed, 29-Jul-2020 10:56:21 GMT
x-cache: MISS from localhost
x-cache-lookup: MISS from localhost:3129
<html>content here</html>
Check out the video below for a visual guide:
Web Unblocker employs pre-defined cookies, headers, and sessions to optimize website unblocking. So, it’s best that you refrain from sending any custom parameters commonly used for unblocking, as they may interfere with Web Unblocker’s ability to get quality data.
That being said, for special cases, we support a list of additional functionalities for target-specific requests:
Parameter | Description |
---|---|
X-Oxylabs-Session-Id |
To reuse the same IP for multiple requests, add a session ID, which can be a random string of characters. |
X-Oxylabs-Geo-Location |
To use an IP address from a specific location. |
Headers | To send custom headers, use x-oxylabs-force-headers: 1 when submitting your request. |
Cookies | To add custom cookies to your requests. |
X-Oxylabs-Status-Code |
If your target site returns a non-standard status code with a successful response, you can send the response status code, and our system will not retry the request. |
X-Oxylabs-Render |
To render JavaScript, use html to get a rendered HTML or png to get a screenshot of the page. You can also define your own browser instructions, such as clicking, inputting, and more. List of instructions. |
POST requests |
Web Unblocker allows you to send a POST request to a web endpoint of your choice. You can send data to a target website, making it return a different result. |
See the following table for a list of some of the most common status codes indicating errors:
Code | Status | Description |
---|---|---|
200 |
OK | All went well. |
400 |
Multiple error messages | Wrong request structure. Perhaps a misspelled parameter or an invalid value. Find more information in the response body. |
401 |
Authorization header not provided/Invalid authorization header/Client not found | Missing authorization header or incorrect login credentials. |
403 |
Forbidden | This resource isn't available to your account. |
404 |
Not Found | The job ID you're looking for is no longer available. |
408 |
Timeout | The request timed out. |
429 |
Too many requests | You've exceeded the rate limit. |
500 , 502 , 503 |
Internal Server Error | We're experiencing some issues. We're probably aware of the issues, but you can report it anyway. |
550 |
Faulted After Too Many Retries | Something went wrong. Try again at no extra cost or get in touch with us. |
If you run into any other response codes, contact our Support Team for assistance.
In the dashboard, you will find a Statistics tab which enables you to track your traffic usage. Here, you’ll find a detailed view of how much traffic you use daily. You can set the time period of usage as preferred and filter the usage by sub-user and domain.
Additionally, you’ll be able to see your requests’ success rates, average response times, and the number of results for each domain.
If you have any further questions, please don't hesitate to get in touch with our Support Team.
Also, feel free to explore other developer-focused resources for more insights:
To ensure that you get the most value out of Web Unblocker, we only charge you for successfully extracted data. This means that failed scraping attempts won’t be included in your traffic usage. Moreover, we calculate both upstream and downstream traffic. For more information on billing, check out our documentation.
To ensure the highest success rates, we implement the auto-retry feature. This means that when a scraping task fails, our system chooses a new set of parameters and resends the request.
Web Unblocker leverages Oxylabs proxy infrastructure, including 102M+ ethically gathered IPs. By combining Residential and Datacenter Proxy pools, Web Unblocker ensures you can effortlessly unlock data in over 195 countries.
Yes, we offer a free trial for our Web Unblocker solution. The trial lasts for one week and you can use up to 1GB (up to 10k results). To claim your free trial, simply register or log in to the dashboard.
No matter if you’re a small business or a large enterprise, we offer a wide variety of payment options. Depending on your data needs, you can choose from Regular and Enterprise tiers, each offering four distinct subscription plans:
Regular:
1-week Free trial (1GB or 10K results) — $0
Micro (5GB) — $75/mo
Starter (25GB) — $325/mo
Advanced (60GB) — $660/mo
Enterprise:
Venture (100GB) — $900/mo
Business (250GB) — $2000/mo
Corporate (500GB) — $3500/mo
Custom+ (1TB+) — $5000/mo
For more information about our pricing, check out our pricing page.
About the author
Vytenis Kaubrė
Technical Copywriter
Vytenis Kaubrė is a Technical Copywriter at Oxylabs. His love for creative writing and a growing interest in technology fuels his daily work, where he crafts technical content and web scrapers with Oxylabs’ solutions. Off duty, you might catch him working on personal projects, coding with Python, or jamming on his electric guitar.
All information on Oxylabs Blog is provided on an "as is" basis and for informational purposes only. We make no representation and disclaim all liability with respect to your use of any information contained on Oxylabs Blog or any third-party websites that may be linked therein. Before engaging in scraping activities of any kind you should consult your legal advisors and carefully read the particular website's terms of service or receive a scraping license.
Yelyzaveta Nechytailo
2024-12-09
Augustas Pelakauskas
2024-12-09
Get the latest news from data gathering world
Scale up your business with Oxylabs®