How to Wait for Selector in Playwright?

Best practices

Use specific and unique selectors to ensure that wait_for_selector accurately targets the desired element on the page.
Set an appropriate timeout for wait_for_selector to avoid unnecessarily long waits, especially in environments with variable network conditions.
Utilize the state parameter in wait_for_selector to handle different visibility states of elements, such as visible (this is set by default), attached, or hidden, according to the needs of your test scenario.
Always handle potential exceptions from wait_for_selector to maintain the robustness of your script, especially in cases where the selector does not appear within the timeout period.

# pip install playwright
from playwright.sync_api import sync_playwright


with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.new_page()
    page.goto('https://sandbox.oxylabs.io/products', wait_until='networkidle')

    # Wait for a specific selector to be visible on the page
    page.wait_for_selector('.product-card h4')
    print(page.query_selector('.product-card h4').text_content())

    # Wait for a selector to be attached to the DOM
    page.wait_for_selector('.product-card h4', state='attached')
    print(page.query_selector('.product-card h4').text_content())

    # Wait for a selector to be hidden or removed
    page.goto('https://sandbox.oxylabs.io/products', wait_until='domcontentloaded')
    page.wait_for_selector('.out-of-stock', state='hidden')
    print(page.query_selector('.out-of-stock'))

    # Optionally, set a timeout (in milliseconds)
    page.wait_for_selector('.product-card h4', timeout=5000)
    print(page.query_selector('.product-card h4').text_content())

    browser.close()

Common issues

Ensure that the page navigation is complete and all data has been loaded before proceeding with further data processing.
Regularly update your selectors if the website undergoes changes to maintain the accuracy and reliability of your scraping operations.
Consider using wait_for_selector with regex or CSS pseudo-classes for dynamic content that changes identifiers frequently.
Test your selectors in the browser's developer tools console to confirm they select the intended elements before implementing them in your script.

url = 'https://sandbox.oxylabs.io/products'

# Incorrect: Not waiting before the page has fully loaded
page.goto(url, timeout=1000)
print(page.query_selector('.in-stock'))

# Correct: Ensure page navigation is complete
page.goto(url, wait_until='networkidle')
print(page.query_selector('.in-stock').text_content())


# Incorrect: Using outdated or incorrect selectors
try:
    page.wait_for_selector('.old-class-name', timeout=5000)
    print(page.query_selector('.old-class-name').text_content())
except:
    print('No element with this selector.')

# Correct: Regularly check and update selectors to match the current page structure
page.wait_for_selector('.product-card h4')
print(page.query_selector('.product-card h4').text_content())


# Incorrect: Using a simple selector for elements that change identifiers
page.wait_for_selector('.css-e8at8d.eag3qlw10')

# Correct: Use selectors that target stable attributes or patterns
page.wait_for_selector('.product-card')


# Incorrect: Not testing selectors in development tools leading to failures in scripts
try:
    page.wait_for_selector('.total-results', timeout=3000)
except:
    print('No such element.')

# Correct: Test selectors in the browser console to ensure they work as expected
page.wait_for_selector('.result-count span')
print(page.query_selector('.result-count span').text_content())