Back to blog
Scraping Data from Etsy: A Comprehensive Guide for Data Extraction


Vytenis Kaubrė
Back to blog
Vytenis Kaubrė
Etsy is an online marketplace focusing on unique items, such as vintage and hand-crafted goods. While it might not directly compete with e-commerce giants like Amazon or eBay, Etsy has grown exponentially in recent years: as of December 31st, 2022, Etsy had over 5.4M active sellers and 89.4M active buyers.
Etsy’s significant user base enables an ideal opportunity for companies and even Etsy sellers to gain value and boost their operations. Thus, web scraping comes in handy in such a case, allowing you to extract Etsy data for:
Price and product availability tracking
Pricing strategy insights
Finding new product trends
Collecting reviews
Competitor research
Finding new leads and potential customers
In this guide, we'll overview why it's challenging to scrape Etsy and what types of public data you can scrape. Next, we'll show you how to implement Etsy scraping with Oxylabs' Etsy Scraper to get the parsed data and save it to a CSV file.
Like most websites these days, Etsy employs anti-scraping measures to stop any bot activity on its website. From tracking the website users’ actions to implementing CAPTCHAs – bypassing these Etsy’s anti-bot measures isn’t a cakewalk.
You may need to use a proxy to constantly change your scraping requests’ IP addresses, as well as set up various headers and user agents to make the requests appear as coming from an actual browser. Managing all of this requires a lot of effort and in-depth knowledge, but a dedicated Etsy web scraping tool like Oxylabs’ Etsy Scraper API can immensely ease the process. It takes care of anti-bot measures with proxies, custom headers, user agents, and other features, allowing you to jump straight to further analysis of data.
With the API, you get a maintenance-free web scraping infrastructure with access to a vast proxy pool. You'll get access to E-Commerce Scraper API (part of Web Scraper API), which enables you to utilize dedicated scrapers for top online marketplaces, including Amazon, eBay, Walmart, and many more.
There are three main types of Etsy web pages:
This is where all the products are listed based on a search term. Etsy product pages include product information, such as:
The product image
The title
Ratings and the number of votes
Price, discount, and the discount end date
And the seller’s name
There you can also find specific tags, like “Star Seller”, “FREE shipping”, “Bestseller”, “Popular now”, and “Etsy’s pick”.
This is a page where you can find all the information about a specific product listing. Product listings include the same information as the product pages, but you’ll also find:
More product images
More information about the seller and the number of sales made
Product highlights
Descriptions
Detailed ratings
Customer reviews
And detailed shipping information
Here, you’ll find more information about the seller and their items. Shop pages include details, like:
The shop title
Short description
Ratings
The number of sales
The profile of the shop’s owner
An announcement
All the items the shop sells
All the customer reviews
A detailed section about the shop
And the shop policies section
Using a web scraping tool, you can scrape Etsy product data as it's publicly available. So, let’s get into it.
Get a free trial to test out Web Scraper API for your project needs.
You must register for an account on the Oxylabs dashboard to use Etsy Scraper API. You can try out the API for 1-week at no cost, but if you’ve already made up your mind – choose a pricing plan that best suits your needs, and follow the steps to sign up and create an account.
Once you create an account, you’ll receive the API credentials from us. You’ll have to use them within your requests.
For demonstration purposes, we’ll use one of the product listings to scrape Etsy product data like the price, title, quantity, etc. The web page looks as follows:
In this tutorial, we’ll use Python and its modules, like requests, csv, and pprint, to make an HTTP request to the API. First, open your terminal and install the requests module:
pip install requests
Create a new Python file and add the following code:
import requests, csv
from pprint import pprint
payload = {
'source': 'universal',
'url': 'https://www.etsy.com/listing/1423148329',
'geo_location': 'United States',
'user_agent_type': 'mobile',
'parse': True,
}
# Get the response.
response = requests.post(
'https://realtime.oxylabs.io/v1/queries',
auth=('user', 'pass'),
json=payload,
)
pprint(response.json())
Replace the url value with the web page you want to scrape data from and the geo_location with the country from which you want to access Etsy. You can set the user_agent_type parameter to any other device of your choice. You can find a list of available user agent parameters in our documentation.
In addition, replace user and pass with the username and password you’ve created in the dashboard. If you wish to add more parameters to your code, visit our documentation for more information. When you run the code, it’ll make a request to the API and return a parsed JSON response:
{
"results": [
{
"content": {
"categories": [
{
"title": "All categories"
},
{
"title": "Art & Collectibles"
},
{
"title": "Fine Art Ceramics"
}
],
"currency": "USD",
"customized": false,
"images": [
"https://i.etsystatic.com/41402376/c/1969/1567/75/243/il/3afc9a/4653235704/il_75x75.4653235704_1g03.jpg",
"https://i.etsystatic.com/41402376/r/il/39f8e3/4701471373/il_75x75.4701471373_q610.jpg",
"https://i.etsystatic.com/41402376/r/il/112b77/4653235192/il_75x75.4653235192_a6vx.jpg"
],
"parse_status_code": 12005,
"price": 618.84,
"product_id": "1423148329",
"seller": {
"best_seller": false,
"reviews_count": 0,
"star_seller": false,
"title": "RoyalSicily",
"url": "https://www.etsy.com/shop/RoyalSicily?ref=simple-shop-header-name&listing_id=1423148329"
},
"shipping": {
"from": "Italy"
},
"stock_status": "Only 1 available",
"title": "Sicilian ceramic lamp Sunflowers 65 cm",
"url": "https://www.etsy.com/listing/1423148329/sicilian-ceramic-lamp-sunflowers-65-cm"
},
"created_at": "2023-02-24 16:17:18",
"job_id": "7034919195095648257",
"page": 1,
"parser_type": "etsy_product",
"status_code": 200,
"updated_at": "2023-02-24 16:17:22",
"url": "https://www.etsy.com/listing/1423148329"
}
]
}
The data extracted and displayed above can be saved into a CSV file using the csv module in Python. Add the following code to your Python file:
# The data is saved in the dictionary
dictionary = response.json()["results"][0]['content']
# The data in the dictionary is then stored in the csv file, "etsy_data.csv"
with open('etsy_data.csv', 'w', newline='') as f:
writer = csv.DictWriter(f, fieldnames=dictionary.keys())
writer.writeheader()
writer.writerow(dictionary)
The JSON response is stored in the dictionary first, and then, using the csv library, we create a file named etsy_data.csv. You can import the CSV to an Excel file for further analysis. And that’s it! You can now tweak or filter the data per your project's requirements.
If you prefer a more manual approach instead of using dedicated web scrapers, you can scrape Etsy data using the requests library in Python along with Oxylabs' Residential Proxies.
Before you start, ensure you have the necessary libraries installed:
pip install requests beautifulsoup4
To avoid getting blocked while scraping Etsy data, use Oxylabs' Residential Proxies. Here's an example of how to make a request through the proxy to scrape a specific Etsy search page:
import requests
from bs4 import BeautifulSoup
import csv
# Configure proxy settings
proxy = {
"http": "http://your_username:your_password@pr.oxylabs.io:7777",
"https": "http://your_username:your_password@pr.oxylabs.io:7777"
}
# Set up headers to mimic a real browser
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
}
# Target URL for scraping
url = "https://www.etsy.com/de-en/search?q=throw+blanket&anchor_listing_id=721378704&ref=hub_stashgrid_module-3&mosv=ssos&moci=1337201102321&mosi=1337201672571&is_merch_library=true&max=75"
# Send the request through the proxy
response = requests.get(url, headers=headers, proxies=proxy)
Once you receive the response, you can parse the HTML using BeautifulSoup to extract specific elements like product titles and prices:
# Check if the request was successful
if response.status_code == 200:
# Parse the HTML content
soup = BeautifulSoup(response.text, "html.parser")
# Find all product information containers
products = soup.find_all("div", class_="v2-listing-card__info")
# Extract data from each product
product_data = []
for product in products:
title = product.find("h3").text.strip() if product.find("h3") else "N/A"
price = product.find("span", class_="currency-value").text.strip() if product.find("span", class_="currency-value") else "N/A"
# Print data for verification
print(f"Title: {title}\nPrice: {price}\n")
# Store data for saving to CSV
product_data.append({"Title": title, "Price": price})
else:
print(f"Failed to scrape: {response.status_code}")
If you wish to save the extracted data, you can store it in a CSV file using Python's csv module:
# Only proceed if we have product data to save
if response.status_code == 200 and product_data:
with open("etsy_search_results.csv", "w", newline="", encoding="utf-8") as file:
# Define CSV writer
fieldnames = ["Title", "Price"]
writer = csv.DictWriter(file, fieldnames=fieldnames)
# Write header and data rows
writer.writeheader()
writer.writerows(product_data)
print("Data saved to etsy_search_results.csv")
When you scrape Etsy listings, remember that Etsy has anti-scraping measures, so rotating Etsy proxies and varying headers can help avoid detection."
If CAPTCHA pages appear frequently, consider using headless browsers like Selenium.
Scrape responsibly and respect the site's terms of service.
Using Oxylabs' Residential Proxies ensures your requests come from real residential IPs, improving the chances of successful data retrieval without being blocked.
If you’re unsure whether to scrape with proxies, without them, or using a dedicated scraper, here’s a comparison table to help you decide.
Approach | Key Features | Advantages | Limitations | Best For |
---|---|---|---|---|
No Proxies | • Basic HTTP requests • Single IP address • Simple request handling |
• Easy to implement • No additional costs • Minimal setup • Small-scale scraping |
• High risk of blocks • Limited request volume • No geo-targeting • Poor scalability |
• Small projects • Non-restricted content • Testing and development |
With proxies | • Rotating IP addresses • Geo-location targeting • Session management |
• High success rates • Scalability • Anti-ban protection • Geographic flexibility |
• Proxy management • More costs • Complex setup • Needs monitoring |
• Large-scale operations • Competitor monitoring • Global data collection |
Scraper APIs | • Pre-built infrastructure • JavaScript rendering • Parsing • CAPTCHA handling |
• Ready-to-use solution • Maintenance-free • Technical support |
• Higher costs • Limited customization • API-specific limitations • Dependency on provider |
• Complex websites • JavaScript-heavy sites • Resource-constrained teams |
Web scraping Etsy product data isn’t easy as it requires a lot of effort to go around the website’s anti-scraping system. You may need to use proxies, rotate different user agents, and pass more complex headers in order to get the data you need on a large scale.
A dedicated marketplace scraper, like Etsy Scraper API, can take away the hardships of web scraping and allow users to scrape the web without a hitch.
Residential Proxies are essential for block-free web scraping. To resemble organic traffic, you can buy proxy solutions, most notably residential and datacenter IPs.
If you have any questions or need assistance, don't hesitate to get in touch with us via email or live chat.
About the author
Vytenis Kaubrė
Technical Copywriter
Vytenis Kaubrė is a Technical Copywriter at Oxylabs. His love for creative writing and a growing interest in technology fuels his daily work, where he crafts technical content and web scrapers with Oxylabs’ solutions. Off duty, you might catch him working on personal projects, coding with Python, or jamming on his electric guitar.
All information on Oxylabs Blog is provided on an "as is" basis and for informational purposes only. We make no representation and disclaim all liability with respect to your use of any information contained on Oxylabs Blog or any third-party websites that may be linked therein. Before engaging in scraping activities of any kind you should consult your legal advisors and carefully read the particular website's terms of service or receive a scraping license.
Etsy, like most websites, employs anti-scraping measures to prevent bot activity on its platform. These include tracking user actions and implementing CAPTCHAs. While Etsy doesn't explicitly allow scraping, public data can be collected using appropriate tools and techniques that respect their systems and terms of service.
To scrape the valuable data about Etsy products effectively, you need tools that can bypass anti-bot measures. A specialized solution like Oxylabs' Etsy Scraper API handles IP rotation, custom headers, and user agents automatically. The process involves setting up an account with a scraping service, creating a request with the target URL and parameters, sending the request to the API, and processing the returned JSON data.
For bulk downloading Etsy listings, you need a scalable scraping solution that can handle multiple requests without getting blocked. Use a dedicated e-commerce scraper with proper proxy rotation. Create a script that iterates through multiple product pages or search results, sends requests for each listing, and saves the structured data to a CSV or database for further analysis.
Web scraping publicly available data is generally legal, but with important caveats. You should scrape only publicly accessible data, respect the website's robots.txt file and terms of service, use the data for legitimate business purposes, avoid excessive requests that could impact the website's performance, and ensure compliance with data protection regulations like GDPR if collecting personal information. It's advisable to consult with a legal professional for your specific use case.
Get the latest news from data gathering world
Scale up your business with Oxylabs®