Back to blog

Python Web Scraping Tutorial: Step-By-Step

We take you through every step of building your first web scraper. Find out how to get started in data acquisition with Python.

Adomas Sulcas

2022-01-06

15 min read

What Is a Proxy Server [2022 Guide]

In this article, we'll answer the most commonly asked questions the web scraping community has on what is a proxy - the most powerful tool for data gathering.

Adomas Sulcas

2022-01-03

13 min read

How to Hide My IP Address

This article aims to broaden one’s knowledge of what an IP address can reveal, and how to hide an IP address. Also, we discuss whether an online business hide their IP address.

Lukas Motiejunas

2022-01-02

6 min read

How Alternative Data Drives E-Commerce Success

Looking to gain a competitive advantage with specialized insights into consumer sentiment, user behavior, and competitor strategies? Alternative data gives you that edge with valuable information from novel sources that include social media websites, mobile application data, search engines, and much more.

Gediminas Rickevicius

2021-12-30

3 min read

Web Scraping With PHP

This article will guide you through the step-by-step process of writing various PHP web scraping routines that can extract public data from static and dynamic web pages.

Augustas Pelakauskas

2021-12-30

11 min read

Introducing Oxy Proxy Manager App

It’s a free proxy app that allows you to add, edit and manage your proxies from any proxy provider of your choice. Learn more and try it out now!

Iveta Vistorskyte

2021-12-28

2 min read

Building a Web Scraper in Golang

This article will guide you through the step-by-step process of writing a fast and efficient Golang web scraper that can extract public data from a target website.

Augustas Pelakauskas

2021-12-23

9 min read

Puppeteer on AWS Lambda

There are a few challenges when it comes to getting Puppeteer to work properly on AWS Lambda, and we’ll address all of them in this post.

Jordan Hansen

2021-12-22

2 min read

How to Automate Competitors' & Benchmark Analysis With Python

The purpose of this article is to help you automate the data extraction processes as much as possible. After learning how to do this, you can dedicate your time to what matters: the analysis itself and coming up with actionable insights to strategize.

Daniel Heredia Mejias

2021-12-22

3 min read

Web Scraping With Ruby

A tutorial that covers the basics of web scraping static and dynamic web pages using Ruby programming language.

Augustas Pelakauskas

2021-12-15

8 min read

Comprehensive Guide on Data Collection

In this extensive white paper, we’ve gathered a variety of technical insights to help you begin web scraping with Python.

Yelyzaveta Nechytailo

2021-12-15

1 min read

E-Commerce Keyword Research: Data Collection Challenges and Solutions

E-commerce keyword research is at the core of every successful e-commerce business. Find out what data collection challenges you may face and how to overcome them.

Iveta Vistorskyte

2021-12-15

7 min read

What Is Data Mining?

Data mining is an advanced analysis of collected datasets. Learn more about data mining techniques, specifics, and benefits

Monika Maslauskaite

2021-12-02

5 min read

What is Browser Fingerprinting?

Browser fingerprinting is being used as a new avenue of tracking. How does it work? Is it possible to reduce the likelihood of being tracked? Read on to find out more.

Adomas Sulcas

2021-12-02

4 min read

Poor Quality Data Might Cost You Too Much

Read a message on the importance of data quality from Allen O’Neill, one of the industry’s leading experts.

Allen O'Neill

2021-12-01

5 min read

Search Engine Scraping: What You Should Know

Want to find out which data sources from search engines are the most beneficial? Did you know that scraping SERPs comes with challenges that can complicate data gathering processes? Read this article and find out everything you need to know about scraping search engines.

Iveta Vistorskyte

2021-11-30

8 min read

How to Extract Data from A Website?

Making data-driven business decisions nowadays is the number one priority for many companies. If you are interested in this field, you should learn how to extract data from websites. Check out!

Iveta Vistorskyte

2021-11-29

8 min read

What Is a Web Session and How Is It Used in Web Scraping?

This article will give you a general overview of web sessions, their relation to cookies, and their use in web scraping.

Augustas Pelakauskas

2021-11-26

5 min read

Aezakmi Proxy Integration With Oxylabs

Use proxies with the Aezakmi anti-detection browser to enhance anonymity. Learn how to integrate Oxylabs’ Datacenter and Residential Proxies in this step-by-step guide.

Monika Maslauskaite

2021-11-16

2 min read

What Is Affiliate Fraud and How to Prevent It?

In this blog post, we’ll discuss affiliate fraud and the most common methods fraudsters use. We’ll also explain how to identify fraud and tips for not falling victim to malicious actors.

Maryia Stsiopkina

2021-11-12

7 min read

Proxy Integration With ParseHub

Extracting data might sometimes be troublesome. To make this process easier, learn how to integrate Oxylabs Residential Proxies with ParseHub tool.

Jolita Pundzaite

2021-11-05

3 min read

Helium Scraper Proxy Integration With Oxylabs

This article will guide you through the integration process of Oxylabs’ Residential Proxies with Helium Scraper.

Augustas Pelakauskas

2021-10-29

2 min read

Proxy Integration With WebHarvy

This article will guide you through the integration process of Oxylabs’ Residential Proxies with WebHarvy’s web scraper.

Augustas Pelakauskas

2021-10-22

2 min read

Playwright Proxy Integration With Oxylabs

In this article, we'll go through the Playwright integration process with Oxylabs’ Residential Proxies.

Iveta Vistorskyte

2021-10-22

2 min read

What Is Sentiment Analysis?

In this article, you will learn what sentiment analysis is, how it can benefit market research and brand monitoring, and how it works.

Maryia Stsiopkina

2021-10-22

10 min read

What Is Data Normalization?

Data normalization is one of the best practices of efficient data management. Find out more about database normalization and how you can benefit from normalizing data in this article.

Monika Maslauskaite

2021-10-18

6 min read

News Scraping: Everything You Need to Know

This article discusses everything you need to know about news scraping, including the benefits and use cases of news scraping as well as how you can use Python to create an article scraper.

Iveta Vistorskyte

2021-10-18

7 min read

Incogniton Integration with Oxylabs

This article will guide you through the integration process of Oxylabs’ Residential Proxies to ensure a smooth takeoff.

Augustas Pelakauskas

2021-10-14

2 min read

Selenium Proxy Integration with Oxylabs

This article will go through the Selenium integration process with Oxylabs’ Residential Proxies for a smooth web scraping process.

Iveta Vistorskyte

2021-10-11

2 min read

Free White Paper: E-commerce Drives ROI Upwards with Alternative Data

Find out how UK ecommerce and retail companies are changing industry practices with alternative data. Dig through the data Oxylabs and Censuswide have collected to see how alternative data has changed ecommerce.

Adomas Sulcas

2021-10-07

1 min read

Next Chapter for Real-Time Crawler: New Products With Dedicated Focus

We have been working on some changes for our Real-Time Crawler product. We are happy to finally announce that starting today, our scraper will be switching its focus and will be split into three distinct Scraper APIs.

Gabija Fatenaite

2021-10-06

3 min read

Free White Paper: Proxies Buying Guide for Enterprises

Learn more about use cases and challenges for which datacenter and residential proxies are best suitable. Find out the key points to consider when choosing a reliable proxy provider.

Monika Maslauskaite

2021-09-27

1 min read

Data Wrangling: What Is It and Why Is It Important?

This article discusses what data wrangling is, the key steps of data wrangling, and why it’s crucial for businesses.

Iveta Vistorskyte

2021-09-23

4 min read

Most Common HTTP Headers

HTTP headers enable to transfer further details within the request or response headers. Find out 5 key HTTP headers that are crucial to use and optimize in web scraping.

Vytautas Kirjazovas

2021-09-20

4 min read

HTTP vs. HTTPS: What Is the Difference?

This article discusses the differences between HTTP and HTTPS protocols, their security parameters, and the steps you should take to switch from HTTP to HTTPS.

Maryia Stsiopkina

2021-09-17

5 min read

Rotating ISP Proxies: Be in Control of Your Scraping Sessions

We at Oxylabs couldn’t wait to share the good news with you - our new product Rotating ISP Proxies is now released! Read this article to learn what makes Rotating ISP Proxies stand out from the rest proxies and how your web scraping projects can benefit from this product by Oxylabs.

Maryia Stsiopkina

2021-09-17

2 min read

13 Tips on How to Crawl a Website Without Getting Blocked

Web crawling and web scraping are essential for data gathering. Getting blacklisted while scraping data is a common issue for those who don’t know how to crawl a website without getting blocked. We gathered a list of actions to prevent getting blacklisted while scraping and crawling websites.

Adelina Kiskyte

2021-09-16

6 min read

Introducing Mobile Proxies: Harness the Power of Mobile IPs

At Oxylabs, we aim to expand our resources to meet even the most demanding business needs. As a result, we constantly grow the number of different types of proxies. Now, we’re excited to introduce Mobile Proxies with an extensive list of locations and country-level & ASN targeting!

Iveta Vistorskyte

2021-09-15

2 min read

What is Firmographic Data? Everything You Need to Know

Firmographic data can provide valuable insight for business-to-business marketers. Read this article to learn what benefits it brings and how it can be acquired.

Maryia Stsiopkina

2021-09-14

6 min read

What Is Parsing of Data?

In this article we’ll dig a little deeper on what is data parsing, and discuss whether building an in-house data parser is more beneficial to a business, or is it better to outsource a data parser.

Gabija Fatenaite

2021-09-13

5 min read

Data-Driven Marketing: How Big Data Helps Make Business Decisions

This article discusses how big data is changing marketing and the process of decision-making. Learn what data-driven marketing is and its main benefits and use cases.

Maryia Stsiopkina

2021-09-02

5 min read

How to Read HTML Tables with Pandas

Read this article to learn everything about pandas library and how useful pandas read_html function can be, especially when combined with other helpful functions.

Iveta Vistorskyte

2021-09-01

7 min read

The Role of Web Scraping in Data-Driven Investing

Data gathering has become an integral part of most, if not all, modern businesses. The degree of importance may vary from business to business, yet in areas such as data-driven investing, it’s the core foundation upon which the entire industry is built.

Iveta Vistorskyte

2021-08-30

1 min read

Reading & Parsing JSON Data With Python: Tutorial

JSON is a common standard used by websites and APIs, and natively supported by modern databases such as PostgreSQL. In this guide, we explain how to handle JSON data with Python.

Monika Maslauskaite

2021-08-30

7 min read

lxml Tutorial: XML Processing and Web Scraping With lxml

Go through the basics of creating XML documents and jump onto processing XML and HTML documents in this Python lxml tutorial.

Gabija Fatenaite

2021-08-30

6 min read

OxyCon 2021: The Top Takeaways From Day Two

The OxyCon 2021 web scraping conference has come to an end but provided us with good food for thought. Read this article summarising the top takeaways from Day Two.

Maryia Stsiopkina

2021-08-27

6 min read

OxyCon 2021: The Top Takeaways From Day One

The first day of OxyCon 2021 has passed. We had some amazing presentations and learned a lot about the world of web scraping. Here are the top key takeaways from day one.

Monika Maslauskaite

2021-08-25

5 min read

Oxylabs Proxy Integration With Puppeteer

Find out more about proxy integration with Puppeteer for efficient dynamic website handling.

Monika Maslauskaite

2021-08-19

2 min read

What Is Financial Data?

This article will discuss different types of financial data, their use cases, and financial data management and analysis tools.

Maryia Stsiopkina

2021-08-18

6 min read

Free White Paper: Web Scraping in the Travel Industry: Main Challenges and Use Cases

Download this free white paper and get all the information on how you can benefit from web scraping in the travel industry and deal with challenges when gathering the required public data.

Iveta Vistorskyte

2021-08-18

1 min read

Top News on Everything Data Gathering

Subscribe to our newsletter and get monthly scraping updates delivered right to your email.

No spam whatsoever, just pure data gathering news, trending topics and useful links. Unsubscribe anytime.

Scale up your business with Oxylabs®