Oxylabs Web API Quick Start Guide

Setting up Web API

1. Register and create an instance

Start by heading to the Oxylabs Dashboard and create an account. If this is your first time with us, you’ll receive a $5 welcome credit to try Web API for free.

Then, click + Add product instance and select Web API. Finally, choose the instance name, if required – set product usage limits, and create the instance.

2. Get your API key

Open your Web API product instance from the dashboard home section. There, look for your new API key and instance username and password at the top of the overview. These credentials are your access to the Web API.

3. Export the API key

Finally, export your API key into your project environment (e.g. your OS terminal or the CLI terminal of the project workspace)

export OXYLABS_WEB_API_KEY=your_api_key_here

Manual Web API Integration

1. Search for relevant URLs

The Web API operates on a core design pattern of search to find, scrape to extract. To start, find which pages hold the answer using POST /v1/search:

curl https://webapi.oxylabs.io/v1/search \
  -H "Authorization: Bearer $OXYLABS_WEB_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query": "eu ai act compliance deadlines", "max_results": 3}'

Response example:

{
  "state": "done",
  "results": [
    {
      "title": "Implementation Timeline | EU Artificial Intelligence Act",
      "overview": "Date 2 August 2025 Providers: need to be compliant by 2 August 2027...",
      "url": "https://artificialintelligenceact.eu/implementation-timeline/",
      "metadata": { "position": 1 }
    }
  ]
}

Learn more about /search endpoint in our documentation.

2. Scrape the page content

Search snippets (overview) returned from the /search endpoint are truncated. To extract the full, fresh content as clean "markdown" (or any preferred format, such as "json", "html", or a "screenshot"), pass the target URL to POST /v1/scrape with the chosen output value:

curl https://webapi.oxylabs.io/v1/scrape \
  -H "Authorization: Bearer $OXYLABS_WEB_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://artificialintelligenceact.eu/implementation-timeline/", "output": ["markdown"]}'

Learn more about /scrape endpoint in our documentation.

Web API AI Integrations

Oxylabs Web API is designed for easy AI agent workflows using Agent Skills, MCP (Model Context Protocol) or simple one-prompt integrations. Once you have your API key, choose the integration method.

1. Model Context Protocol (MCP) Server

Self-host our official MCP server to expose multiple Web API tools using /search and /scrape endpoints directly to Claude Code, Cursor, or any MCP-friendly AI agent:

uv tool install git+https://github.com/oxylabs/web-api-mcp

To learn more about Web API MCP integration and its tools, see our documentation.

2. Agent Skills

Add official agent skills that teach your AI agent how to search, scrape, cite sources, and handle web research through Oxylabs Web API:

npx skills add oxylabs/web-api-skills

Now your agent knows all there is to know about Oxylabs Web API and how to use it. To learn more about Web API agent skills, see our documentation.

3. One-Page Integration Prompt

If you want an AI coding assistant (like Cursor or Copilot) to build a whole integration for you straight from the chat, paste the following prompt:

"Integrate the Oxylabs Web API into this project for live web search and page reading. Read https://developers.oxylabs.io/products/web-api/for-agents.md and follow its reference code and retry rules. The API key is in OXYLABS_WEB_API_KEY. Execute a test search, scrape the top result as Markdown, and verify error handling."

To see a full example and learn more about Web API integration, see our documentation.

Dedicated Scrapers

Beyond the universal URL scraper (POST /v1/scrape), the Web API provides numerous scrapers for specific targets under POST /v1/scrape/{target}/{collection}. That includes search engines, AI & LLM sources, ecommerce sites, and media platforms.

POST /v1/scrape/amazon/product
{ "query": "B0935DN1BN", "output": ["json"], "domain": "de", "location": "10115", "currency": "EUR" }

GET /v1/scrapers returns an up-to-date list of all currently available dedicated scrapers, which collect structured, pre-parsed JSON data.

Learn more about all available targets and their parameters in the API Reference in our documentation.

AI Parsing

You can extract structured fields from any web page in the same request by setting output: ["json"] and providing a prompt or an OpenAPI schema under the json parameter.

Prompt example:

{
  "url": "https://sandbox.oxylabs.io/products/1",
  "output": ["json"],
  "json": {
    "prompt": "Parse the product title, price as a number, currency"
  }
}

OpenAPI Schema example:

{
  "url": "https://sandbox.oxylabs.io/products/1",
  "output": ["json"],
  "json": {
    "schema": {
      "type": "object",
      "properties": {
        "title": { "type": "string" },
        "price": { "type": "number" },
        "in_stock": { "type": "boolean" }
      },
      "required": ["title", "price"]
    }
  }
}

Data Delivery

By default, every request above runs in a synchronous realtime method, where the result comes back in the response. Once you're working with large-scale tasks with a chance of client timeout, we recommend asynchronous methods.

Common Response Codes & Retries

/scrape

Usage statistics

The Usage section of your product instance lets you see detailed information on your Web API usage in any time range and check per-endpoint statistics. You can see usage graphs for any selected period and use finer filters.

Additional Resources

• Documentation

• Agent skills

• MCP server

• YouTube

• Blog