Common reasons

  • Government-imposed censorship or restrictions on specific types of content.

  • Legal injunctions that specifically prevent certain information from being accessible.

  • Copyright or intellectual property rights enforcement.

  • Compliance with local laws prohibiting the display or distribution of certain content.

Scrollable code block. Use arrow keys to scroll.

Try Scraper API with 5K results

Methods to try

  • Review and optimize the frequency of requests to ensure they are spread out evenly over time.

  • Implement exponential backoff in your request retry strategy to reduce the likelihood of hitting the rate limit.

  • Increase the rate limit by upgrading your service plan or negotiating higher limits with the service provider.

  • Utilize caching mechanisms to reduce the number of requests made to the server.

Scrollable code block. Use arrow keys to scroll.

Leave HTTP errors with our API

Web Scraper API

Self-Service

Public data delivery from the majority of websites

From

49

Useful resources

Python request library
Python Requests Library: 2026 Guide
Authors avatar

Adomas Sulcas

2025-01-02

Laptop and books on table
What are HTTP Headers: Examples and Most Common Types
Mantas Miksenas avatar

Mantas Miksenas

2024-07-10

WebSocket vs HTTP
WebSocket vs HTTP: The Differences
Maryia Stsiopkina avatar

Maryia Stsiopkina

2021-08-18

Get the latest news from data gathering world