Scraping Data from Etsy: A Comprehensive Guide for Data Extraction

Etsy is an online marketplace focusing on unique items, such as vintage and hand-crafted goods. While it might not directly compete with e-commerce giants like Amazon or eBay, Etsy has grown exponentially in recent years. As of December 31st, 2022, Etsy had over 5.4M active sellers and 89.4M active buyers, according to Marketplace Pulse.

Etsy’s significant user base enables an ideal opportunity for companies and even Etsy sellers to gain value and boost their operations. Thus, web scraping comes in handy in such a case, allowing you to extract Etsy data for:

• Price and product availability tracking

• Pricing strategy insights

• Finding new product trends

• Collecting reviews

• Competitor research

• Finding new leads and potential customers

In this guide, we’ll overview why it’s challenging to scrape Etsy and what types of public data you can scrape. Next, we’ll show you how to scrape Etsy product data with Oxylabs’ Etsy Scraper to get the parsed data and save it to a CSV file.

Difficulties of scraping Etsy's marketplace

Like most websites these days, Etsy employs anti-scraping measures to stop any bot activity on its website. From tracking the website users’ actions to implementing CAPTCHAs – bypassing these Etsy’s anti-bot measures isn’t a cakewalk. 

You may need to use a proxy to constantly change your scraping requests’ IP addresses, as well as set up various headers and user agents to make the requests appear as coming from an actual browser. Managing all of this requires a lot of effort and in-depth knowledge, but a dedicated web scraper like Oxylabs’ Etsy Scraper API can immensely ease the process. It takes care of anti-bot measures with proxies, custom headers, user agents, and other features, allowing you to jump straight to further analysis of data.

With the API, you get a maintenance-free web scraping infrastructure with access to a vast proxy pool. You'll get access to E-Commerce Scraper API (part of Web Scraper API), which enables you to utilize dedicated scrapers for top online marketplaces, including Amazon, eBay, Walmart, and many more.

Overview of Etsy’s page types

There are three main types of Etsy web pages: 

Product pages

This is where all the products are listed based on a search term. Etsy product pages include product information, such as:

• The product image

• The title

• Ratings and the number of votes

• Price, discount, and the discount end date

• And the seller’s name

There you can also find specific tags, like “Star Seller”, “FREE shipping”, “Bestseller”, “Popular now”, and “Etsy’s pick”.

Product listings

This is a page where you can find all the information about a specific product listing. Product listings include the same information as the product pages, but you’ll also find:

• More product images

• More information about the seller and the number of sales made

• Product highlights

• Descriptions

• Detailed ratings

• Customer reviews

• And detailed shipping information

Shop pages

Here, you’ll find more information about the seller and their items. Shop pages include details, like:

• The shop title

• Short description

• Ratings

• The number of sales

• The profile of the shop’s owner

• An announcement

• All the items the shop sells

• All the customer reviews

• A detailed section about the shop

• And the shop policies section

Using a web scraping tool, you can scrape Etsy product data as it's publicly available. So, let’s get into it.

In this tutorial, we’ll use Python and its modules, like requests, csv, and pprint, to make an HTTP request to the API. First, open your terminal and install the requests module:

pip install requests

Step 3: Send the request

Create a new Python file and add the following code:

import requests, csv
from pprint import pprint


payload = {
   'source': 'universal',
   'url': 'https://www.etsy.com/listing/1423148329',
   'geo_location': 'United States',
   'user_agent_type': 'mobile',
   'parse': True,
}


# Get the response.
response = requests.post(
   'https://realtime.oxylabs.io/v1/queries',
   auth=('user', 'pass'),
   json=payload,
)


pprint(response.json())

Replace the url value with the web page you want to scrape data from and the geo_location with the country from which you want to access Etsy. You can set the user_agent_type parameter to any other device of your choice. You can find a list of available user agent parameters in our documentation.

In addition, replace user and pass with the username and password you’ve created in the dashboard. If you wish to add more parameters to your code, visit our documentation for more information. When you run the code, it’ll make a request to the API and return a parsed JSON response:

{
  "results": [
    {
      "content": {
        "categories": [
          {
            "title": "All categories"
          },
          {
            "title": "Art & Collectibles"
          },
          {
            "title": "Fine Art Ceramics"
          }
        ],
        "currency": "USD",
        "customized": false,
        "images": [
          "https://i.etsystatic.com/41402376/c/1969/1567/75/243/il/3afc9a/4653235704/il_75x75.4653235704_1g03.jpg",
          "https://i.etsystatic.com/41402376/r/il/39f8e3/4701471373/il_75x75.4701471373_q610.jpg",
          "https://i.etsystatic.com/41402376/r/il/112b77/4653235192/il_75x75.4653235192_a6vx.jpg"
        ],
        "parse_status_code": 12005,
        "price": 618.84,
        "product_id": "1423148329",
        "seller": {
          "best_seller": false,
          "reviews_count": 0,
          "star_seller": false,
          "title": "RoyalSicily",
          "url": "https://www.etsy.com/shop/RoyalSicily?ref=simple-shop-header-name&listing_id=1423148329"
        },
        "shipping": {
          "from": "Italy"
        },
        "stock_status": "Only 1 available",
        "title": "Sicilian ceramic lamp Sunflowers 65 cm",
        "url": "https://www.etsy.com/listing/1423148329/sicilian-ceramic-lamp-sunflowers-65-cm"
      },
      "created_at": "2023-02-24 16:17:18",
      "job_id": "7034919195095648257",
      "page": 1,
      "parser_type": "etsy_product",
      "status_code": 200,
      "updated_at": "2023-02-24 16:17:22",
      "url": "https://www.etsy.com/listing/1423148329"
    }
  ]
}

Step 4: Save the data to CSV

The data extracted and displayed above can be saved into a CSV file using the csv module in Python. Add the following code to your Python file:

# The data is saved in the dictionary
dictionary = response.json()["results"][0]['content']

# The data in the dictionary is then stored in the csv file, "etsy_data.csv"
with open('etsy_data.csv', 'w', newline='') as f:
    writer = csv.DictWriter(f, fieldnames=dictionary.keys())
    writer.writeheader()
    writer.writerow(dictionary)

The JSON response is stored in the dictionary first, and then, using the csv library, we create a file named etsy_data.csv. You can import the CSV to an Excel file for further analysis. And that’s it! You can now tweak or filter the data per your project's requirements.

Wrapping up

Web scraping Etsy product data isn’t easy as it requires a lot of effort to go around the website’s anti-scraping system. You may need to use proxies, rotate different user agents, and pass more complex headers in order to get the data you need on a large scale.

A dedicated web scraper, like Etsy Scraper API, can take away the hardships of web scraping and allow users to scrape the web without a hitch.

Residential Proxies are essential for block-free web scraping. To resemble organic traffic, you can buy proxy solutions, most notably residential and datacenter IPs.

If you have any questions or need assistance, don't hesitate to get in touch with us via email or live chat.