Use specific and unique CSS selectors to improve the accuracy and performance of element retrieval.
Opt for class selectors (`.class-name`) for elements with common styles, and ID selectors (`#id-name`) for unique elements, ensuring faster and more reliable element selection.
When selecting elements based on attributes, ensure to use the attribute value that uniquely identifies the element, like `[type='submit']` for buttons.
Combine CSS selectors to navigate DOM hierarchies, such as `.parent-class .child-class`, to target specific elements within a nested structure efficiently.
# pip install selenium from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC # Initialize the WebDriver (using Chrome in this example) driver = webdriver.Chrome() # Make sure you have ChromeDriver installed try: # Navigate to the webpage driver.get("https://sandbox.oxylabs.io/products") # Wait for page to load wait = WebDriverWait(driver, 10) # Select element by CSS class product_name = wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, ".title"))) print(product_name.text.strip()) # Select element by element tag product_name = driver.find_element(By.TAG_NAME, "h4") print(product_name.text.strip()) # Select elements by attribute (full class attribute) price = driver.find_element(By.CSS_SELECTOR, "[class='price-wrapper css-li4v8k eag3qlw4']") print(price.text.strip()) # Combining CSS selectors info = driver.find_element(By.CSS_SELECTOR, ".category span") print(info.text.strip()) finally: # Close the browser driver.quit()
Ensure your CSS selectors are updated to match any changes in the webpage's structure or style to avoid stale element reference errors.
Avoid using overly complex CSS selectors which can lead to slower execution times and harder maintenance; simplify where possible.
Use pseudo-classes like `:first-child`, `:last-child`, or `:nth-child(n)` to target specific elements in a list or among siblings for more precise control.
Regularly test and validate your CSS selectors in the browser's developer tools before implementing them in your Selenium scripts to catch any potential issues early.
# pip install selenium from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait driver = webdriver.Chrome() driver.get("https://sandbox.oxylabs.io/products") wait = WebDriverWait(driver, 10) try: # Bad example: Using outdated CSS selector try: old_price = driver.find_element(By.CSS_SELECTOR, ".old-price-label") print(old_price.text.strip()) except Exception as e: print(e) # Good example: Updated CSS selector after webpage changes current_price = driver.find_element(By.CSS_SELECTOR, ".price-wrapper") print(current_price.text.strip()) # Bad example: Overly complex CSS selector complex_selector = driver.find_element(By.CSS_SELECTOR, "div.price-wrapper.css-li4v8k.eag3qlw4") print(complex_selector.text.strip()) # Good example: Simplified CSS selector simple_selector = driver.find_element(By.CSS_SELECTOR, ".price-wrapper") print(simple_selector.text.strip()) # Bad example: Selecting the first item without using pseudo-classes first_item = driver.find_elements(By.CSS_SELECTOR, ".category span")[0] print(first_item.text.strip()) # Good example: Using :first-child pseudo-class first_item = driver.find_element(By.CSS_SELECTOR, ".category span:first-child") print(first_item.text.strip()) # Bad example: Not testing CSS selector in browser before use try: untested_selector = driver.find_element(By.CSS_SELECTOR, ".unverified-selector") print(untested_selector.text.strip()) except Exception as e: print(e) # Good example: Manually verify selector works in browser's developer tools first verified_selector = driver.find_element(By.CSS_SELECTOR, "h4") print(verified_selector.text.strip()) finally: driver.quit()
Web scraper API
Public data delivery from a majority of websites
From
49
Get the latest news from data gathering world
Scale up your business with Oxylabs®
Proxies
Advanced proxy solutions
Data Collection
Datasets
Resources
Innovation hub