Get structured data in JSON from a list of ready-to-use data APIs
Extract data from any website with HTML Crawler API
100% delivery from search engines and e-commerce websites
Receive structured data in JSON from ready-to-use data APIs with focus on search engines and e-commerce sites.
Carry out web crawling projects for most websites in HTML without getting blocked for more resource-efficient data gathering.
Get data in the most convenient way.
We will render JS for you
{
"title": "See Real-Time Crawler in action!",
"message": "Enter your keyword to see the real output example.",
"note": "Choose other criteria (optional).",
}
With Real-Time Crawler e-commerce API, get parsed data for:
Real-Time Crawler search engine API provides parsed data for:
HTML Crawler API provides raw data with added features such as:
Pay only for successful pages*
Extract data from most websites without getting blocked
Smooth data gathering ensured by Next-Gen Residential Proxies powered by AI/ML algorithms
Patented Oxylabs Proxy Rotator allows to achieve successful requests considerably faster
Get structured JSON data in real-time or via callback method from leading e-commerce and search engine sites
Supports high volumes of requests by utilizing Oxylabs global proxy infrastructure
Tailored requests on country & city level, or by device
Resilient to website changes
Handles IP blocks and CAPTCHAs
Takes care of proxy management
import requests
from pprint import pprint
# Structure payload.
payload = {
'source': 'universal',
'url': 'https://stackoverflow.com/questions/tagged/python',
'user_agent_type': 'desktop',
}
# Get response.
response = requests.request(
'POST',
'https://realtime.oxylabs.io/v1/queries',
auth=('user', 'pass1'),
json=payload,
# This will return the JSON response with results.
pprint(response.json())
<?php
$params = array(
'source' => 'universal',
'query' => 'https://stackoverflow.com/questions/tagged/python',
'user_agent_type' => 'desktop',
);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://realtime.oxylabs.io/v1/queries");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($params));
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_USERPWD, "user" . ":" . "pass1");
$headers = array();
$headers[] = "Content-Type: application/json";
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$result = curl_exec($ch);
echo $result;
if (curl_errno($ch)) {
echo 'Error:' . curl_error($ch);
}
curl_close ($ch);
?>
curl --user user:pass1 'https://realtime.oxylabs.io/v1/queries' -H "Content-Type: application/json"
-d '{"source": "universal", "url": "https://stackoverflow.com/questions/tagged/python", "user_agent_type": "desktop"}'
https://realtime.oxylabs.io/v1/queries?source=universal&url=https%3A%2F%2Fstackoverflow.com%2Fquestions%2Ftagged%2Fpython&user_agent_type=desktop&access_token=1234abcd
Country and ASN filtering
100% delivery
Highly scalable
Starter
99
Includes one of the following:
60K
Pages in HTML
40K
Pages in HTML with JS rendering
29K
E-commerce/search engine API pages
Top-up prices:
$1.65
/1000 pages in HTML
$2.50
/1000 pages in HTML with JS rendering
$3.50
/1000 pages for e-commerce/search engine API
Business
399
Includes one of the following:
285K
Pages in HTML
190K
Pages in HTML with JS rendering
160K
E-commerce/search engine API pages
Top-up prices:
$1.40
/1000 pages in HTML
$2.10
/1000 pages in HTML with JS rendering
$2.50
/1000 pages for e-commerce/search engine API
Corporate
999
Includes one of the following:
833K
Pages in HTML
555K
Pages in HTML with JS rendering
526K
E-commerce/search engine API pages
Top-up prices:
$1.20
/1000 pages in HTML
$1.80
/1000 pages in HTML with JS rendering
$1.90
/1000 pages for e-commerce/search engine API
Enterprise
Starts from:
10,000
Includes one of the following:
14M+
Pages in HTML
11M+
Pages in HTML with JS rendering
10M+
E-commerce/search engine API pages
Advice on target scraping
Parsed data
Patented proxy rotator
No proxy maintenance
Highly customizable
24/7 live support
Leave your contact details
Receive a personal offer
Start collecting data or begin a free trial
You will receive an email with more details within one working day. Your personal Sales Manager will be happy to schedule a call with you and help you set up the proxies for your specific use case.
