Configure several schedules for different jobs
Automation is key when it comes to web data acquisition. If you have identical scraping jobs that you execute on a regular basis, let Scheduler do the work for you. Create multiple jobs, use the Cron expression to set the time intervals, and get automatic data updates at ease.
Get data automatically delivered to your cloud storage
Enjoy peace of mind knowing that with scheduled jobs, you’ll receive updates and data to your preferred cloud storage automatically and regularly.
Receive notifications once the data is transferred
To receive regular data updates, you’ll no longer need to send new requests and create or maintain scheduling scripts. Just set up recurring scraping jobs, and we’ll do the rest – once the scraped and parsed data is sent to your cloud storage, you’ll receive a notification.
How does Scheduler work?
Scheduler is a feature of our Web Scraper API for setting up recurring scraping and parsing jobs. Its process can be broken down into three parts: you set up the frequency using Cron, provide the scraping and parsing parameters, and set the end time.
Step 1
Set a frequency rate at which we’ll be executing the scraping jobs.
Step 2
Provide us with the job parameter sets of your choice.
Step 3
Get a notification as soon as the result is in your cloud storage.
Step 4
Set the end time and we’ll stop delivering the data.
Web Scraper API with the Scheduler feature
Scheduler is an additional feature our Scraper API users get for free.
Web Scraper API delivers real-time data from most websites, including search engines, e-commerce marketplaces, and much more.
Customizable request parameters
JavaScript rendering
Structured data in JSON
City-level results without IP blocks
Jorūnė Skridailaitė
Senior Account Manager @ Oxylabs
By scraping and parsing data according to a set interval, Scheduler does the heavy lifting – the data is sent to the client’s cloud storage so they don’t need to send identical requests repeatedly.
Vincent Patrizio
Enterprise Account Manager @ Oxylabs
Scheduler is there to simplify the scraping and parsing jobs you do routinely. According to your preference, it can execute scraping and parsing jobs every minute, hour, or day.
A word from your Dedicated Account Manager
Get personal assistance from a Dedicated Account Manager regarding any web scraping-related matter.
Useful resources
Step-By-Step Tutorial
Follow this detailed video guide to getting started with Scheduler.
Where to Start & How it Works
Read this article to learn more about automating your web scraping and parsing jobs.
In-depth Tutorial
Check this detailed tutorial with code examples on how to use Scheduler.
Frequently Asked Questions
What is Scheduler?
Scheduler is a free feature of Oxylabs’ Web Scraper API. It allows you to automate regular and identical scraping jobs as well as receive data at the frequency of your choice. This way, you no longer need to send new requests or create scheduling scripts to create repetitive scraping and parsing jobs.
How to set up jobs with Scheduler?
You have to submit a request for us to create a schedule, where you specify:
How often should we repeat the job(s)
What exactly is the job
Job end time
Callback URL (optional)
Cloud storage URL (optional)
If you provide us with the callback URL, we’ll send you a notification once the scheduled job is done. You’ll be able to access the scraped and parsed results in your cloud storage (if you specified it.)
Get the latest news from data gathering world
Scale up your business with Oxylabs®