Most Common User Agents For Price Scraping

Iveta Vistorskyte

Last updated by Augustas Pelakauskas

2025-04-04

4 min read

In a fast-changing business world, data extraction is highly needed for market research, so the concept of web scraping becomes more and more known to many. To take a bigger slice of the market, information is one of the most important components for businesses. Data collection can be very time consuming, so by automating the whole process with web scraping, businesses can focus on other tasks.

Pricing information is important for businesses that want to be competent players in the market. It helps to shape the whole strategy and adjust prices against their competitors.

Are you considering price scraping for your company? There are a few web scraping challenges you should know about. Complicated web page structures, CAPTCHA, login requirements, IP blocking, and more. In this article, we’ll explain how to avoid being blocked by target servers. Let’s learn about user agents and how they relate to price scraping.

If you are interested in web scraping and have web scraping project ideas, don’t forget to check out our other blog posts.

What is user agent?

Did you know everyone currently browsing the web has a user agent? The definition of a user agent indicates that it serves as a user representative on the internet.

The user agent acts as the bridge between the user and the internet. Imagine needing to specify information about your browser, operating system, software, and device type every time you browse any website. Surfing the internet would be very complicated and time consuming. This is the reason why every browser has a user agent.

Here is an example of a browser’s user agent string:

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_2) AppleWebKit/605.1.15 (KHTML, like Gecko) Safari/605.1.15 Version/13.0.4

When your browser connects to a website, the user agent string is included in the HTTP header. Why does the website need information about the user? The web server uses this information to adapt the content to specific web browsers and operating systems.

How to find my user agent in any browser?

Most popular web browsers use the same principle.

Open Developer Tools (press F12 on Windows or Option + ⌘ + I on macOS).
In the Console tab, input the following line to see your current browser’s user agent string: navigator.userAgent.

Google Chrome

Google Chrome user agent string

Mozilla Firefox

Mozilla Firefox user agent string

Microsoft Edge

Microsoft Edge user agent string

Safari

Safari user agent string

How to switch my user agent in any browser: Chrome, Firefox, Edge, Safari?

Google Chrome

Open Developer Tools > ⋮ > More tools > Network conditions > Uncheck Use browser default.
Choose or create a new user agent string.

1. Go to Network conditions

2. Uncheck the default setting

3. Set up a custom user agent string

Mozilla Firefox

Type in about:config and search for general.useragent.override.
If not found, create a new one as a String and input a custom user agent string.

1. Input about:config

2. Search for general.useragent.override

3. Set up a new user agent string

Microsoft Edge

Open Developer Tools > ⋮ > Network conditions > Uncheck Use browser default.
Choose or create a new user agent string.

1. Open settings

2. Go to Network conditions

3. Uncheck the default setting

4. Set up a custom user agent string

Safari

As of late (2025), Safari doesn’t allow the browser user agent string to be changed natively. You have to find a third-party app. User agent switcher Safari or a similar search keyword will eventually get you the right solution.

Examples of different user agents for desktop browsers and mobile devices

Different types of user agents are commonly used based on the device (desktop, mobile), operating system, browser, or bot (what is a bot?).

Each browser user agent string can provide valuable insights, which can be important for server configuration, handling requests, and providing an optimized user experience. Below are the examples of common user agent types.

Desktop user agents

Desktop user agents are used by web browsers running on desktop operating systems like Windows, macOS, and Linux.

Chrome on Windows

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.85 Safari/537.36

Safari on macOS

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Version/13.1.2 Safari/537.36

Mobile user agents

Mobile user agents are used by mobile devices like smartphones and tablets, running on operating systems like Android or iOS.

Safari on iOS

Mozilla/5.0 (iPhone; CPU iPhone OS 14_4_1 like Mac OS X) AppleWebKit/537.36 (KHTML, like Gecko) Version/14.0 Mobile/15E148 Safari/537.36

Chrome on Android

Mozilla/5.0 (Linux; Android 11; Pixel 4 Build/RQ1A.210205.003) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.91 Mobile Safari/537.36

Bot user agents

User agents from web crawlers and bots, such as Googlebot, Bingbot, or other search engines, often identify themselves as bots to avoid being blocked. Also, AI tools crawl the web, gathering data to generate answers.

Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

User agents for price scraping

Price scraping is one of the most important types of web scraping for every business. It helps e-commerce companies follow the real-time selling prices of products on their competitors’ sites.

Of course, some websites block any scraping because, for example, they do not believe in open data access. There are more ways to block web scraping, but one is to block user requests from user agents that don’t belong to the main browsers. It’s one of the first checks that allows data sources to identify suspicious requests.

When web scraping is in progress, the web server receives numerous requests. If user agents, for example, are identical, the web server identifies requests as suspicious activity. Most web scrapers don’t bother to change their user agents, but as you now understand, it’s crucial.

Also, you should remember to keep the user agents up to date because every browser or operating system changes user agents.

Price Scraping is one of the most important types of web scraping for every business

Most common user agents for price scraping

There is no such thing as special user agents for price scraping. As you already know, it’s crucial to use the most popular user agents for web scraping because this is one of the ways to avoid being blocked by the data resource server. If you use obsolete or rare user agents, there is a big chance that a web server will identify a web scraping process as suspicious, and you may be blocked.

Also, if you are searching for the best web scraping tools for your business, check out Oxylabs’ Web Scraper API for a week for free.

Wrap up

In short, a user agent acts as the bridge between the user and the internet. It gives the web server necessary information about your browser, software, device type, etc. According to this information, web servers can display different web pages for you.

Setting up the most common user agents for price scraping reduces the chances of being blocked by targeted servers because it’s one of the first checks that allows websites to identify questionable requests.

Frequently asked questions

What is web scraping?

Web scraping is an automated process in which you extract public web data and import the information on your device or into any local file. Web scraping has become an essential tool for business development.

What is price scraping?

Price scraping is the extraction of price data using a web scraper. The workflow consists of searching and copying data from websites to be analyzed later. Even if it sounds simple and you can do it yourself, price scraping tools help save a lot of time, especially if you need to extract data from many websites. Then, the only thing left is to analyze the found data. This information helps businesses with pricing strategy, including promos, discounts, special offers, etc.

What are the most popular user agents?

The most popular browser user agents are Chrome (for both computers and phones), Safari (for iPhones and Macs), Firefox, Edge, and Samsung Internet. Chrome has the most users worldwide, while Safari comes in second since it's what most Apple devices use. Firefox and Edge have decent followings, too, though smaller, and Samsung Internet is pretty common on Samsung phones.

What is a user agent identifier?

A user agent identifier is a supplement various websites use to simplify user agent identification. Nowadays, it’s difficult to determine what a given user agent string represents because automated bots, mobile devices, and desktop web browsers have exploded into many different forms. The user agent identifier is an up-to-date database containing the latest user agent and bot signatures.

What is Web Scraper API?

Web Scraper API is a heavy-duty data extraction tool built for data extraction from most websites, ensuring a high data delivery success rate.

About the author

Iveta Vistorskyte

Lead Content Manager

Iveta Vistorskyte is a Lead Content Manager at Oxylabs. Growing up as a writer and a challenge seeker, she decided to welcome herself to the tech-side, and instantly became interested in this field. When she is not at work, you'll probably find her just chillin' while listening to her favorite music or playing board games with friends.

Learn more about Iveta Vistorskyte

All information on Oxylabs Blog is provided on an "as is" basis and for informational purposes only. We make no representation and disclaim all liability with respect to your use of any information contained on Oxylabs Blog or any third-party websites that may be linked therein. Before engaging in scraping activities of any kind you should consult your legal advisors and carefully read the particular website's terms of service or receive a scraping license.