Best practices

  • Use implicit waits for general delays, such as when elements may take a moment to load.
    However, avoid using them for actions that need precise timing or specific conditions.

  • Use explicit waits when you need to wait for something specific – like an element to appear, become clickable, or disappear. You can define exactly how long to wait and what condition must be met.

  • Don’t mix implicit and explicit waits. Combining them can cause unpredictable delays and make your script harder to troubleshoot.

  • Always set a timeout when using explicit waits. This prevents your script from hanging forever if the condition is never met.

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException

# Initialize WebDriver
driver = webdriver.Chrome()

# Set implicit wait
driver.implicitly_wait(10) # Waits up to 10 seconds before throwing a NoSuchElementException

# Navigate to a webpage
driver.get("https://sandbox.oxylabs.io/products")

# Example of explicit wait: wait until an element is clickable
try:
element = WebDriverWait(driver, 20).until(
EC.element_to_be_clickable((By.ID, "target-element-id"))
)
element.click()
except TimeoutException:
print("Element not clickable within 20 seconds")

# Clean up: close the browser
driver.quit()

Common issues

  • Ensure that the element locators used in explicit waits are accurate and unique to avoid targeting the wrong elements or multiple elements unintentionally.

  • Handle exceptions, especially TimeoutException. If the wait times out, catch the error and show a helpful message or try an alternative action.

  • Regularly update the conditions used in explicit waits to align with changes in the web application's UI and functionality to maintain test reliability.

  • Use explicit waits with AJAX-loaded content. These waits help ensure that dynamic elements are fully loaded before you try to interact with them.

# Incorrect: Using a non-unique ID that might match multiple elements
element = WebDriverWait(driver, 20).until(
EC.element_to_be_clickable((By.ID, "non-unique-id"))
)

# Correct: Using a unique and specific locator
element = WebDriverWait(driver, 20).until(
EC.element_to_be_clickable((By.ID, "unique-element-id"))
)

# Incorrect: Not handling exceptions, which might leave the browser hanging if an element is not found
element = WebDriverWait(driver, 20).until(
EC.element_to_be_clickable((By.ID, "some-id"))
)
element.click()

# Correct: Handling TimeoutException to provide feedback or take alternative actions
try:
element = WebDriverWait(driver, 20).until(
EC.element_to_be_clickable((By.ID, "some-id"))
)
element.click()
except TimeoutException:
print("Element not clickable within 20 seconds")

# Incorrect: Using outdated or incorrect conditions that no longer match the UI
element = WebDriverWait(driver, 20).until(
EC.visibility_of_element_located((By.ID, "old-ui-element"))
)

# Correct: Regularly updating the wait conditions to match the current UI elements
element = WebDriverWait(driver, 20).until(
EC.visibility_of_element_located((By.ID, "updated-ui-element"))
)

# Incorrect: Trying to interact with AJAX-loaded elements without waiting for them to load
driver.get("https://example.com")
element = driver.find_element(By.ID, "ajax-element")
element.click()

# Correct: Using explicit waits for AJAX-loaded elements to ensure they are fully loaded
driver.get("https://example.com")
element = WebDriverWait(driver, 20).until(
EC.presence_of_element_located((By.ID, "ajax-element"))
)
element.click()

Try Oxylabs' Proxies & Scraper API

Residential Proxies

Self-Service

Human-like scraping without IP blocking

From

8

Datacenter Proxies

Self-Service

Fast and reliable proxies for cost-efficient scraping

From

1.2

Web scraper API

Self-Service

Public data delivery from a majority of websites

From

49

Useful resources

Puppeteer vs Selenium: Which to Choose
author avatar

Yelyzaveta Hayrapetyan

2025-05-27

Web Scraping with Selenium and Python
Gabija Fatenaite avatar

Gabija Fatenaite

2024-06-13

Scrapy vs. Selenium: Which One is Better?
Scrapy vs. Selenium: Which One is Better?
vytenis kaubre avatar

Vytenis Kaubrė

2023-07-12

Get the latest news from data gathering world

I'm interested