Across 2019, dozens of developments have been added to our ever-evolving Real-Time Crawler. All of these additions have been designed to improve existing functionality and help our partners to increase the efficiency and productivity of their daily web scraping operations.
With less than a month in 2020, we are proud to announce that Real-Time Crawler sees a further enhancement, which will bring additional value to our partners.
From now on, our partners can pass real-time request parameters via URL, and retrieve the data instantly on the same connection, either in raw HTML or parsed in JSON. This enhancement will result in further ease of use of our solution and save additional time when setting up web scraping assignments to Real-Time Crawler.
In simplest terms, our partners will be able to specify their exact requirements without writing a single line of code. The one-line integration is based on the real-time endpoint, meaning the data will be fetched back instantly within seconds on the same request.
Please note: Data retrieval time will vary depending on the data source that is being used. For further information about the new feature, please refer to our official documentation or get in touch with your personal account manager.
What is Real-Time Crawler, and why consider It?
The web is an insightful but complex source of knowledge, as public data sources frequently change their structure. Some data-wealthy sites require advanced technical knowledge to gain access, while others add additional steps that mess up any data-gathering operation. That is why the majority of our clients turn to our Real-Time Crawler solution to gather public data from the web at scale and do so in real-time.
In short, Oxylabs Real-Time Crawler is a robust data gathering tool optimized for effortless data extraction from search engines and e-commerce sites, also referred to as real-time web scraping solution. In fact, if it caught your attention, check out our in-depth blog post on Real-Time Crawler and its advantages.
Or, if you would like to get familiarized with our solution in video format, hit the play button below to hear what our Product Owner Aleksandras has to say:
Real-Time Crawler will no doubt see further additions in 2020, but already our partners are benefitting from being able to retrieve accurate web data in a hassle-free manner on demand. Hence, if you would like to see it in action, please fill in your contact details or feel free to drop us a line via live chat or email us at [email protected].