Data acquisition

Follow this detailed tutorial and learn how to combine cURL with Python for effective data transfer processes.

Learn what Data as a Service (DaaS) is, how it works, what role it plays in business development, and what to consider when choosing your own DaaS provider.

An extensive Scrapy Splash tutorial on scraping public web data from JavaScript-rendered websites.

Discover the essentials of Web Unblocker and learn how to get started with it.

If CAPTCHAs keep on interrupting your day-to-day scraping tasks, read this article presenting solutions that can help you go around them successfully.

This article covers everything you need to get started with Crawlee. Learn more about its benefits and see a working example of scraping a website with Crawlee.

Web Crawler allows you to explore and collect data quickly and efficiently using our maintenance-free infrastructure.

Learn all about static vs. rotating proxies, sticky IP addresses, and proxy implications in product information scraping at large scale in this blog post.

This white paper provides a comprehensive overview of how AI and its subfield, machine learning, shape the current trends in web scraping.

Find out about the critical features of HTTPX, Requests, and AIOHTTP and how they differ from each other.

Dive deeper into the features, benefits, and drawbacks of Puppeteer and Selenium to make an informed decision on which tool fits you best.

Follow this tutorial to scrape and parse the HTML of a desired website. Learn about the PyQuery library, how to use it in Python, and how it differs from Beautiful Soup.

Find out how Playwright and Puppeteer stack up against each other in particular scenarios, including web scraping.

Let's discuss Playwright vs Selenium, their relevance to web scraping, and what to remember when picking one for your scraping task.

Go is becoming increasingly popular due to its many strengths, but how does it compare to Python, the most common and widely used language in programming?

Learn how an antidetect browser helps with using several accounts for various digital initiatives.

Find out the numerous features of proxies and how they improve security.

Check this step-by-step tutorial to learn how to scrape data from websites into Excel. Find out Excel's capabilities of extracting information from various websites.

In this guide, you will learn how to scrape websites using Google Sheets, and you will find out about the functions in use and how to fix common errors.

ETL stands for extract, transform, and load. It’s an independent three-stage process that moves data from one source(s) to a database and is crucial for modern day anlysis.

Find out what screen scraping is and how it works. Learn the difference between web scraping and screen scraping.

Planning a project on web scraping and don’t know where to start? Learn how proxies come into play when project planning for web scraping in our most recent article.

As part of the combined efforts of Oxylabs and OTA Insight to shed light on the developments in the web scraping industry, we sat down with Glen De Cauwsemaeker, the Lead Crawler Engineer at OTA Insight, to talk about the challenges when scaling data acquisition solutions.

In this tutorial, we will focus on describing the asynchronous approach to scraping multiple URLs and by comparing it to the synchronous one, demonstrate why it can be more beneficial.

In this practical tutorial, you'll learn how to create a web scraper using Rust and collect public product data from an e-commerce website.

Learn what a data pipeline architecture is, why it's important for businesses and how to build one.

Businesses are complex, and both first and second movers encounter unique difficulties and advantages. If you're interested in how these advantages relate to web scraping, check this out!

Learn more about how to effectively track and measure data quality with the help of this detailed blog post

Let's take a look at the key factors influencing data acquisition costs, and discuss ways to reduce these expenses.

This white paper will walk you through the critical stages of the online media monitoring process.

This white paper aims to guide you through the process of large-scale data gathering with an emphasis on e-commerce.

A step-by-step tutorial on making HTTP requests in Node.js with Fetch API, detailing the actionable code and a comparison to Axios.

Oxylabs, in cooperation with Censuswide, has surveyed over 1000 senior decision makers in the finance industry. Find out how alternative data and web scraping has changed finance.

At one point or another, any scaling or coding project will run into the issue of scaling and how to effectively do it. In the 4th episode of OxyCast, we will explore topics such as horizontal vs. vertical scaling, bottleneck avoidance, and many others!

This white paper provides an action chain for price monitoring, from collecting target URLs to data parsing, along with tips and explanations on the most important elements and processes.

Parsing is an integral part of any web scraping activity that helps businesses get the data they need in the right format. In the third episode of OxyCast, we will dig deeper into data parsing and discuss such topics as easy vs. hard parsing, selectors, parser failures, and the future of parsing.

As public web data collection and analysis grow more integral to many companies across various industries, automated data gathering tools prove themselves as indispensable assistants in this challenging task. Check out this white paper to learn more!

The role of alternative data becomes more and more prominent. However, its collection is tied to multiple technological challenges that can disturb your business's operations. This white paper provides a detailed explanation of these challenges and proposes solutions to deal with them.

If you’ve ever tried web scraping, you should be aware of the blocking issue. It’s a common challenge, especially if you gather public data on a large scale without a decent knowledge of using resources wisely. This is why we decided to cover this topic and share our knowledge and tips & tricks on how to avoid getting blocked.

Read insights from an expert in the data collection field on implementing the ethical policy in your company.

This white paper will go through four most popular Python libraries and the basics on how to get started in web scraping.

In this extensive white paper, we’ve gathered a variety of technical insights to help you begin web scraping with Python.

Browser fingerprinting is being used as a new avenue of tracking. How does it work? Is it possible to reduce the likelihood of being tracked? Read on to find out more.

Read a message on the importance of data quality from Allen O’Neill, one of the industry’s leading experts.

Want to find out which data sources from search engines are the most beneficial? Did you know that scraping SERPs comes with challenges that can complicate data gathering processes? Read this article and find out everything you need to know about scraping search engines.

Making data-driven business decisions nowadays is the number one priority for many companies. If you are interested in this field, you should learn how to extract data from websites. Check out!

This article will give you a general overview of web sessions, their relation to cookies, and their use in web scraping.

This article discusses everything you need to know about news scraping, including the benefits and use cases of news scraping as well as how you can use Python to create an article scraper.

Learn more about use cases and challenges for which datacenter and residential proxies are best suitable. Find out the key points to consider when choosing a reliable proxy provider.

HTTP headers enable to transfer further details within the request or response headers. Find out 5 key HTTP headers that are crucial to use and optimize in web scraping.