Best practices

  • Use specific and unique CSS selectors to improve the accuracy and performance of element retrieval.

  • Opt for class selectors (`.class-name`) for elements with common styles, and ID selectors (`#id-name`) for unique elements, ensuring faster and more reliable element selection.

  • When selecting elements based on attributes, ensure to use the attribute value that uniquely identifies the element, like `[type='submit']` for buttons.

  • Combine CSS selectors to navigate DOM hierarchies, such as `.parent-class .child-class`, to target specific elements within a nested structure efficiently.

from selenium import webdriver
from selenium.webdriver.common.by import By

# Initialize WebDriver
driver = webdriver.Chrome()

# Navigate to a webpage
driver.get("https://sandbox.oxylabs.io/products")

# Select element by CSS class
product_name = driver.find_element(By.CSS_SELECTOR, ".product-name")
print(product_name.text)

# Select element by ID
product_id = driver.find_element(By.CSS_SELECTOR, "#product-id")
print(product_id.text)

# Select elements by attribute
price = driver.find_element(By.CSS_SELECTOR, "[data-price='value']")
print(price.text)

# Combining CSS selectors
info = driver.find_element(By.CSS_SELECTOR, ".product-info .info-details")
print(info.text)

# Close the browser
driver.quit()

Common issues

  • Ensure your CSS selectors are updated to match any changes in the webpage's structure or style to avoid stale element reference errors.

  • Avoid using overly complex CSS selectors which can lead to slower execution times and harder maintenance; simplify where possible.

  • Use pseudo-classes like `:first-child`, `:last-child`, or `:nth-child(n)` to target specific elements in a list or among siblings for more precise control.

  • Regularly test and validate your CSS selectors in the browser's developer tools before implementing them in your Selenium scripts to catch any potential issues early.

# Bad example: Using outdated CSS selector
old_price = driver.find_element(By.CSS_SELECTOR, ".old-price-label")

# Good example: Updated CSS selector after webpage changes
current_price = driver.find_element(By.CSS_SELECTOR, ".current-price-label")

# Bad example: Overly complex CSS selector
complex_selector = driver.find_element(By.CSS_SELECTOR, "body > div > div > div > ul > li > a")

# Good example: Simplified CSS selector
simple_selector = driver.find_element(By.CSS_SELECTOR, ".menu-item > a")

# Bad example: Selecting the first item without using pseudo-classes
first_item = driver.find_elements(By.CSS_SELECTOR, ".item-list li")[0]

# Good example: Using :first-child pseudo-class
first_item = driver.find_element(By.CSS_SELECTOR, ".item-list li:first-child")

# Bad example: Not testing CSS selector in browser before use
untested_selector = driver.find_element(By.CSS_SELECTOR, ".unverified-selector")

# Good example: Manually verify selector works in browser's developer tools first
verified_selector = driver.find_element(By.CSS_SELECTOR, ".verified-selector")

Try Oxylabs' Proxies & Scraper API

Residential Proxies

Self-Service

Human-like scraping without IP blocking

From

8

Datacenter Proxies

Self-Service

Fast and reliable proxies for cost-efficient scraping

From

1.2

Web scraper API

Self-Service

Public data delivery from a majority of websites

From

49

Useful resources

Get the latest news from data gathering world

I'm interested