How to Scrape Google Images With Python

What data can be scraped from Google Images?

You can scrape a wide range of useful information from Google Images. The following types of information can be extracted from Google Images:

• Image URLs: The URLs of the individual images are the most often extracted data from Google images. These URLs can be used to access and download photos that are hosted on different websites.

• Image titles: You may learn from this image metadata how images are tagged and described online.

• Thumbnails for images: Images' thumbnails can also be scraped. Smaller copies of the images that are frequently seen in search results or on websites are known as thumbnails.

• Source URLs: Besides the image URLs, you can scrape the URLs of the pages where the images are located. This can add context and additional information to the images.

Image captions and descriptions: Many images in Google images have informative names or captions. Scraping these captions can reveal information on the subject matter and setting of the pictures.

These are a few examples of gathering Google Images data. The specific data you aim to obtain will depend on your goals and the insights you're seeking. Extracting this information can be beneficial, but it's important to always consider and respect the copyright and usage rights associated with the images you collect.

Scrape Google Images using Oxylabs’ Google Images Scraper API

For this tutorial, we will use Google Images Search API to get the Google images related to the one given in the query. This Google Image scraper helps us retrieve all the related images and the URLs (where these images are hosted).  

To use this API, you must create an account on the Oxylabs dashboard and get the API credentials. These credentials will be used in the later stages.

1. Set up the environment

To get started, you must have Python 3.6+ installed and running on your system. Begin by creating a virtual environment and activating it. On Unix devices, for example, you can run these two commands in your terminal:

python3 -m venv .venv
source .venv/bin/activate

You’ll need the following Python packages to put the code into action:

• Requests - for sending HTTP requests to Oxylabs API.

• Pandas - for saving our output data in dataframes and then saving in CSV files. 

Install all these packages in your active environment with the following command:

pip install requests pandas 

2. Import the required libraries

After the installation of packages, create a new Python file and import the required libraries using the following code:

import requests
import pandas as pd

3. Structure the payload

The Google Image Scraper API has the source, query, and context parameters that are mandatory if you want to extract Google Images. All other parameters, such as geo_location, parse, pages, etc., are optional and allow you to modify the result according to your needs. The details of these and other parameters can be found in the official documentation. 

The payload is structured as follows:

# Request payload with API parameters
payload = {
    "source": "google_search",
    "query": "cute cat",
    "geo_location": "United States", # Localize results for US
    "context": [
        {"key": "tbm", "value": "isch"}, # Scrapes only Google Images
    ],
    "parse": True, # Automatically parse HTML into JSON
    "pages": 2 # Scrape as many pages as needed
}

Make sure to replace the query parameter value with the search term for which you want to find relevant images. 

You may also use the Advanced Google Search Operators to filter results. For example, to find images that are only present on Unsplash, modify your query parameter value to cute cats inurl: unsplash.

The parse parameter is set to True to automatically parse the results and receive them in structured JSON format. Additionally, you can use pages and start_page parameters to scrape multiple result pages starting from the start_page. A value of 1 is the default value for both parameters.

4. Make the request

After creating the payload structure, you can initiate a POST request to Oxylabs’ API using the following code segment:

# Use your Oxylabs Web Scraper API credentials
USERNAME = "your_API_username"
PASSWORD = "your_API_password"

# Send a request to Oxylabs Web Scraper API
response = requests.post(
   "https://realtime.oxylabs.io/v1/queries",
   auth=(USERNAME, PASSWORD),
   json=payload
)

# Print the response
print(response.json())

Make sure to use your Web Scraper API credentials. The response received can be viewed in the JSON format. 

5. Extract the data and save it in a CSV file

You can extract the required images from the response.json() object. The response object has a key results that contains all the related image data. We will extract and save all the image data in the data frame. Later, this dataframe can be saved in a CSV file using the following code.

# Get the response data
response_data = response.json()

all_images = []
# Loop through each page in the results
for page in response_data["results"]:
    # Get the organic results from each page
    organic_results = page["content"]["results"]["organic"]
    # Extract image data from each organic result
    for image in organic_results:
        all_images.append({
            "title": image.get("title", ""),
            "link": image.get("link", ""),
            "image": image.get("image", ""),
            "domain": image.get("domain", ""),
            "position": image.get("pos", ""),
            "position_overall": image.get("pos_overall", "")
        })

# Create a DataFrame directly from the extracted data
df = pd.DataFrame(all_images)

# Save to CSV
df.to_csv("google_images.csv", index=False)
print("Successfully saved all images.")

This code saves only the organic search results. If you expect paid/sponsored image results, make sure to add logic that additionally saves the paid results from the API response.

Complete Google Images scraper example

Let’s put all the code together:

import requests
import pandas as pd


USERNAME = "your_API_username"
PASSWORD = "your_API_password"

payload = {
    "source": "google_search",
    "query": "cute cat",
    "geo_location": "United States",
    "context": [
        {"key": "tbm", "value": "isch"},
    ],
    "parse": True,
    "pages": 2
}

response = requests.post(
   "https://realtime.oxylabs.io/v1/queries",
   auth=(USERNAME, PASSWORD),
   json=payload
)

response_data = response.json()

all_images = []
for page in response_data["results"]:

    organic_results = page["content"]["results"]["organic"]

    for image in organic_results:
        all_images.append({
            "title": image.get("title", ""),
            "link": image.get("link", ""),
            "image": image.get("image", ""),
            "domain": image.get("domain", ""),
            "position": image.get("pos", ""),
            "position_overall": image.get("pos_overall", "")
        })

df = pd.DataFrame(all_images)
df.to_csv("google_images.csv", index=False)
print("Successfully saved all images.")

Executing the code will output a CSV file that contains all the Google Image data scraped from two pages:

Conclusion

Google image scraping without a dedicated tool is a complex task. As such, since Google Images as a repository offers a vast and diverse collection that's invaluable for various applications and analyses, implementing a solution like Oxylabs' Google Scraper can be key.

Looking to scrape data from other Google sources? See our in-depth guides for scraping Google search results or Jobs, Scholar, Trends, News, Flights, Shopping, and Maps data.

For web scraping, proxies are an essential anti-blocking measure. To avoid detection by the target website, you can buy proxies of various types to fit any Google image scraping scenario.