Automated Web Scraper With Python & Windows Task Scheduler

In this article, you’ll learn how to set up Windows Task Scheduler to schedule a Python web scraping script automatically & periodically.

Augustas Pelakauskas

2022-08-04

5 min read

Automating Web Scraping With Python and Cron

If you're curious about how to automate your web scraping projects with Python and cron, then check out this dedicated tutorial for more information.

Danielius Radavicius

2022-07-29

6 min read

OxyCon® 2022 Agenda is Live - How to Get the Most Out of the Event

This year's OxyCon has already garnered attention from every corner of the data gathering industry. Now is the time to reveal what's in store during our two-day event on September 7-8th.

Danielius Radavicius

2022-07-28

2 min read

Marketplace SEO: How to Improve Your SERP Rankings with Data

The success of your SEO strategy will likely depend entirely on how accurately you follow a specific marketplace’s regulations and in what ways you stand out.

Danielius Radavicius

2022-07-26

7 min read

Machine Learning: The Driving Force of Web Scraping | OxyCast #6

A brand new episode of OxyCast is live! This time, our favorite host Augustinas Kalvis (Software Developer), and a special guest Jurijus Gorskovas (Machine Learning Engineer), delve deeper into the world of Machine Learning! Watch it to understand the details of machine learning and how it can make web scraping processes more efficient.

Iveta Liupševičė

2022-07-13

3 min read

The Second Mover’s Advantage

Businesses are complex, and both first and second movers encounter unique difficulties and advantages. If you're interested in how these advantages relate to web scraping, check this out!

Julius Černiauskas

2022-07-05

4 min read

Data Quality Metrics You Should Track and Measure

Learn more about how to effectively track and measure data quality with the help of this detailed blog post

Yelyzaveta Hayrapetyan

2022-06-27

6 min read

Brand New Lessons on Everything Data Gathering

Data gathering is a rather tricky field; therefore, getting advice and lessons from people who have worked at the top of the industry for years is unparalleled.

Danielius Radavicius

2022-06-22

2 min read

How to Estimate and Reduce Data Collection Costs

Let's take a look at the key factors influencing data acquisition costs, and discuss ways to reduce these expenses.

Maryia Stsiopkina

2022-06-22

7 min read

Oxylabs Cooperates with CODE

Oxylabs partnered with the University of Michigan, ranked as the No.1 public university in the United States, and School of Information professor Christopher Brooks to share expertise in the field of ethical web scraping.

Gabija Birgile

2022-06-16

2 min read

Real-Time Online Media Monitoring Infrastructure

This white paper will walk you through the critical stages of the online media monitoring process.

Maryia Stsiopkina

2022-06-15

1 min read

OxyCon® 2022: Leading-Edge Conference in All Things Web Scraping

Don’t forget to save your seat at OxyCon 2022, September 7-8, to join discussions on the most relevant and recently encountered topics of public data gathering! Just like last year, the 2 day virtual event will feature discussions from a plethora of industry-leading guests.

Danielius Radavicius

2022-06-09

2 min read

Hard Data vs. Soft Data: The Difference

While being so different, hard and soft data can complement one another in significant ways when it comes to business data analysis and forecasting. Read this blog post to learn more.

Maryia Stsiopkina

2022-05-25

8 min read

Proxies for Web Scraping: a Complete Guide | OxyCast #5

In the 5th episode of our podcast, OxyCast host and Software Engineer Augustinas Kalvis will be talking to Mindaugas Dunderis about proxies and how they go hand-in-hand with public data scraping.

Vytenis Kaubrė

2022-05-23

3 min read

Free White Paper: Alternative Data Unlocks Key Decisions in the UK And US Finance Industries

Oxylabs, in cooperation with Censuswide, has surveyed over 1000 senior decision makers in the finance industry. Find out how alternative data and web scraping has changed finance.

Adomas Sulcas

2022-05-04

2 min read

Web Scraping With RegEx

RegEx stands for Regular Expressions, a method to match specific patterns depending on the provided combinations, which can be used as filters to get the desired output.

Augustas Pelakauskas

2022-04-29

3 min read

Scaling: Overcoming Your Limits | OxyCast #4

At one point or another, any scaling or coding project will run into the issue of scaling and how to effectively do it. In the 4th episode of OxyCast, we will explore topics such as horizontal vs. vertical scaling, bottleneck avoidance, and many others!

Danielius Radavicius

2022-04-25

3 min read

Empowering the Lithuanian Public Sector in the Mission for a Cleaner Internet

The internet is full of illegal and harmful content, which can be hard to detect without the right tools. Web scraping is a perfect solution.

Erika Jankevičė

2022-03-31

2 min read

Real-Time Price Monitoring System Architecture

This white paper provides an action chain for price monitoring, from collecting target URLs to data parsing, along with tips and explanations on the most important elements and processes.

Augustas Pelakauskas

2022-03-24

1 min read

Data Parsing: The Basic, the Easy, and the Difficult | OxyCast #3

Parsing is an integral part of any web scraping activity that helps businesses get the data they need in the right format. In the third episode of OxyCast, we will dig deeper into data parsing and discuss such topics as easy vs. hard parsing, selectors, parser failures, and the future of parsing.

Yelyzaveta Hayrapetyan

2022-03-23

2 min read

Scraping Alternative Data: Technological Challenges to Keep in Mind

The role of alternative data becomes more and more prominent. However, its collection is tied to multiple technological challenges that can interrupt your business's operations. This white paper provides a detailed explanation of these challenges and proposes solutions to deal with them.

Yelyzaveta Hayrapetyan

2022-03-10

1 min read

Using Data for Competitive Advantage

Learn how companies outperform their competitors through the different uses of big data.

Vytenis Kaubrė

2022-03-07

5 min read

Web Scraping: Another Block In The Wall | OxyCast #2

If you’ve ever tried web scraping, you should be aware of the blocking issue. It’s a common challenge, especially if you gather public data on a large scale without a decent knowledge of using resources wisely. This is why we decided to cover this topic and share our knowledge and tips & tricks on how to avoid getting blocked.

Iveta Liupševičė

2022-02-22

2 min read

Web Scraping for Machine Learning

This tutorial explores a real-life scenario where web scraping and machine learning work in tandem and how the step-by-step process should look like if you decide to do it on your own.

Danielius Radavicius

2022-02-22

7 min read

What is MAP Monitoring?

Minimum Advertised Price (MAP) monitoring is an essential process for any brand, supplier or manufacturer that works with e-commerce marketplaces. Tracking MAP policy compliance plays an important role ensuring fair competition across different channels and protecting your brand’s reputation.

Adelina Kiskyte

2022-02-11

4 min read

The Importance of Having an Ethical Data Collection Policy

Read insights from an expert in the data collection field on implementing the ethical policy in your company.

Cornelius (Con) Conlon

2022-02-03

4 min read

OxyCast: A New Podcast on Everything Web Scraping Related

Web scraping is such a broad topic, and there’s a lot of things to learn in order to collect the required public data efficiently. That’s why our team decided to start a new podcast on everything web scraping related – OxyCast!

Iveta Liupševičė

2022-01-20

2 min read

Best Python Libraries for Web Scraping

This white paper will go through four most popular Python libraries and the basics on how to get started in web scraping.

Yelyzaveta Hayrapetyan

2022-01-12

1 min read

AI/ML in 2022: More Real-world Deployments, Focus on AI Ethics and Blockchain Hopes

Oxylabs' AI/ML Advisory Board members make their predictions on what's in store for artificial intelligence and machine learning in 2022.

Adomas Sulcas

2022-01-10

3 min read

Introducing Oxy® Proxy Manager App

It’s a free proxy app that allows you to add, edit and manage your proxies from any proxy provider of your choice. Learn more and try it out now!

Iveta Liupševičė

2021-12-28

2 min read

Building a Web Scraper in Golang

This article will guide you through the step-by-step process of writing a fast and efficient Golang web scraper that can extract public data from a target website.

Augustas Pelakauskas

2021-12-23

9 min read

How to Automate Competitors' & Benchmark Analysis With Python

The purpose of this article is to help you automate the data extraction processes as much as possible. After learning how to do this, you can dedicate your time to what matters: the analysis itself and coming up with actionable insights to strategize.

Daniel Heredia Mejias

2021-12-22

4 min read

Puppeteer on AWS Lambda

There are a few challenges when it comes to getting Puppeteer to work properly on AWS Lambda, and we’ll address all of them in this post.

Jordan Hansen

2021-12-22

3 min read

Comprehensive Guide on Data Collection

In this extensive white paper, we’ve gathered a variety of technical insights to help you begin web scraping with Python.

Yelyzaveta Hayrapetyan

2021-12-15

1 min read

E-Commerce Keyword Research: Data Collection Challenges and Solutions

E-commerce keyword research is at the core of every successful e-commerce business. Find out what data collection challenges you may face and how to overcome them.

Iveta Liupševičė

2021-12-15

7 min read

What Is Data Mining?

Data mining is an advanced analysis of collected datasets. Learn more about data mining techniques, specifics, and benefits

Monika Maslauskaite

2021-12-02

5 min read

Poor Quality Data Might Cost You Too Much

Read a message on the importance of data quality from Allen O’Neill, one of the industry’s leading experts.

Allen O'Neill

2021-12-01

6 min read

Search Engine Scraping: What You Should Know

Want to find out which data sources from search engines are the most beneficial? Did you know that scraping SERPs comes with challenges that can complicate data gathering processes? Read this article and find out everything you need to know about scraping search engines.

Iveta Liupševičė

2021-11-30

8 min read

What Is a Web Session and How Is It Used in Web Scraping?

This article will give you a general overview of web sessions, their relation to cookies, and their use in web scraping.

Augustas Pelakauskas

2021-11-26

5 min read

What Is Affiliate Fraud and How to Prevent It?

In this blog post, we’ll discuss affiliate fraud and the most common methods fraudsters use. We’ll also explain how to identify fraud and tips for not falling victim to malicious actors.

Maryia Stsiopkina

2021-11-12

8 min read

Proxy Integration With ParseHub

Extracting data might sometimes be troublesome. To make this process easier, learn how to integrate Oxylabs Residential Proxies with ParseHub tool.

Jolita Pundzaitė

2021-11-05

4 min read

What Is Sentiment Analysis?

In this article, you will learn what sentiment analysis is, how it can benefit market research and brand monitoring, and how it works.

Maryia Stsiopkina

2021-10-22

10 min read

News Scraping: Everything You Need to Know

This article discusses everything you need to know about news scraping, including the benefits and use cases of news scraping as well as how you can use Python to create an article scraper.

Iveta Liupševičė

2021-10-18

7 min read

What Is Data Normalization?

Data normalization is one of the best practices of efficient data management. Find out more about database normalization and how you can benefit from normalizing data in this article.

Monika Maslauskaite

2021-10-18

6 min read

Free White Paper: E-commerce Drives ROI Upwards with Alternative Data

Find out how UK ecommerce and retail companies are changing industry practices with alternative data. Dig through the data Oxylabs and Censuswide have collected to see how alternative data has changed ecommerce.

Adomas Sulcas

2021-10-07

1 min read

Free White Paper: Proxies Buying Guide for Enterprises

Learn more about use cases and challenges for which datacenter and residential proxies are best suitable. Find out the key points to consider when choosing a reliable proxy provider.

Monika Maslauskaite

2021-09-27

2 min read

Data Wrangling: What Is It and Why Is It Important?

This article discusses what data wrangling is, the key steps of data wrangling, and why it’s crucial for businesses.

Iveta Liupševičė

2021-09-23

4 min read

Most Common HTTP Headers

HTTP headers enable to transfer further details within the request or response headers. Find out 5 key HTTP headers that are crucial to use and optimize in web scraping.

Vytautas Kirjazovas

2021-09-20

4 min read

HTTP vs. HTTPS: What Is the Difference?

This article discusses the differences between HTTP and HTTPS protocols, their security parameters, and the steps you should take to switch from HTTP to HTTPS.

Maryia Stsiopkina

2021-09-17

5 min read

Introducing Mobile Proxies: Harness the Power of Mobile IPs

We’re excited to introduce Mobile Proxies with an extensive list of locations, as well as country, state, city, coordinate, and ASN targeting with no extra fees.

Iveta Liupševičė

2021-09-15

2 min read