How to scroll down or up using Selenium?

Learn how to effectively scroll up or down on web pages using Selenium in this concise tutorial. Master this essential skill to enhance your data extraction techniques and overcome common automation challenges.

Best practices

Use `Keys.PAGE_DOWN` and `Keys.PAGE_UP` for quick scrolling, which simulates keyboard actions and is useful for pages that respond to keyboard events.
Employ `driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")` to scroll directly to the bottom of the page, ensuring you reach all content loaded dynamically.
Utilize `driver.execute_script("window.scrollTo(0, 0);")` to return to the top of the page, which is effective for resetting the view after extensive page navigation.
To focus on a specific element, use `driver.execute_script("arguments[0].scrollIntoView();", element)` which aligns the element to the visible area of the browser window, enhancing element interaction reliability.

from selenium import webdriver
from selenium.webdriver.common.keys import Keys

# Initialize the driver
driver = webdriver.Chrome()

# Open the target website
driver.get("https://sandbox.oxylabs.io/products")

# Scroll down using the PAGE_DOWN key
driver.find_element_by_tag_name('body').send_keys(Keys.PAGE_DOWN)

# Scroll up using the PAGE_UP key
driver.find_element_by_tag_name('body').send_keys(Keys.PAGE_UP)

# Scroll to the bottom of the page using JS
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

# Scroll to the top of the page using JS
driver.execute_script("window.scrollTo(0, 0);")

# Scroll to a specific element
element = driver.find_element_by_id('specificElementId')
driver.execute_script("arguments[0].scrollIntoView();", element)

# Close the driver
driver.quit()

Common issues

Ensure that the element you want to scroll to is loaded before executing the scroll command to avoid `NoSuchElementException`.
Adjust the scrolling speed by using a loop with `window.scrollBy(0, y)` in JavaScript, where `y` is a small value, to mimic human-like scrolling.
When using `Keys.PAGE_DOWN` or `Keys.PAGE_UP`, verify that the focus is on a scrollable area or element to prevent ineffective scrolling actions.
For dynamic web pages, periodically check for the presence of new elements or additional content after scrolling to ensure all elements are interacted with properly.

# Incorrect: Trying to scroll to an element that might not be loaded yet
element = driver.find_element_by_id('dynamicElementId')
driver.execute_script("arguments[0].scrollIntoView();", element)

# Correct: Ensure element is present before scrolling
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

wait = WebDriverWait(driver, 10)
element = wait.until(EC.visibility_of_element_located((By.ID, 'dynamicElementId')))
driver.execute_script("arguments[0].scrollIntoView();", element)

// Inorrect: Scrolling too quickly to the bottom of the page
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

// Correct: Scroll gradually to mimic human scrolling
for (let y = 0; y < document.body.scrollHeight; y += 100) {
setTimeout(() => {
window.scrollBy(0, 100);
}, 100);
}

# Incorrect: Using PAGE_DOWN without ensuring focus is on a scrollable element
driver.send_keys(Keys.PAGE_DOWN)

# Correct: Ensure focus is on a scrollable element before using PAGE_DOWN
body = driver.find_element_by_tag_name('body')
body.click() # Focus on the body or a specific scrollable element
body.send_keys(Keys.PAGE_DOWN)

# Incorrect: Assuming all content is loaded after a single scroll
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

# Correct: Periodically check for new elements after scrolling
import time

driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
time.sleep(3) # Wait for potential dynamic content to load
# Check if more content is available and scroll again if necessary