Use explicit waits (e.g., WebDriverWait) to handle dynamic content that might not be immediately available when the page loads.
Always validate your XPath expressions using browser developer tools before integrating them into your scripts.
Be cautious with contains() in XPath – it can unintentionally match multiple elements. Refine your expressions to be as specific as possible.
Keep your Selenium WebDriver and browser updated to maintain compatibility and leverage new features and bug fixes.
from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC # Initialize WebDriver driver = webdriver.Chrome() # Navigate to the page driver.get("https://sandbox.oxylabs.io/products") # Method 1: Using XPath to find element by exact text element = driver.find_element(By.XPATH, "//tagname[text()='Exact Text']") # Method 2: Using XPath with contains() for partial text match partial_element = driver.find_element(By.XPATH, "//tagname[contains(text(), 'Part of Text')]") # Method 3: Using WebDriverWait to find element by text with timeout wait_element = WebDriverWait(driver, 10).until( EC.presence_of_element_located((By.XPATH, "//tagname[contains(text(), 'Timely Text')]")) ) # Close the browser driver.quit()
Ensure that the text used in XPath expressions is not dynamically generated, as this can lead to element not found errors.
Avoid using absolute XPaths; instead, opt for relative paths to increase the robustness and maintainability of your tests.
Overusing text() can reduce flexibility – attributes (like id or class) are often more stable.
Text matches are case-sensitive in XPath. If case may vary, use translate() for a case-insensitive match.
# Incorrect: Using dynamically generated text in XPath element = driver.find_element(By.XPATH, "//div[text()='Session ID: 12345']") # Correct: Use static parts of the text or attributes element = driver.find_element(By.XPATH, "//div[contains(text(), 'Session ID:')]") # Incorrect: Using absolute XPath element = driver.find_element(By.XPATH, "/html/body/div[1]/div[2]/div[3]/div") # Correct: Using relative XPath element = driver.find_element(By.XPATH, "//div[@class='specific-class']") # Incorrect: Overusing text() where attributes could be used element = driver.find_element(By.XPATH, "//button[text()='Click me']") # Correct: Targeting attributes element = driver.find_element(By.XPATH, "//button[@id='submit-button']") # Incorrect: Ignoring case sensitivity in text searches element = driver.find_element(By.XPATH, "//div[text()='welcome']") # Correct: Using translate() to handle case variations element = driver.find_element(By.XPATH, "//*[translate(text(), 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz')='welcome']")
Web scraper API
Public data delivery from a majority of websites
From
49
Get the latest news from data gathering world
Scale up your business with Oxylabs®
Proxies
Advanced proxy solutions
Data Collection
Datasets
Resources
Innovation hub