Hermes Agent is a pluggable agent framework that integrates web search and extraction through swappable backends. The hermes-web-oxylabs plugin registers Oxylabs AI Studio as one of available backends, calling AI Studio's REST API directly (no vendor SDK) to provide the agent with:
Ranked web search results – title, URL, description
Clean page data as markdown, for research, comparisons, and follow-up reasoning
Fresh web context for more accurate answers
Make sure you have the following before starting:
Hermes Agent installed with hermes CLI
Oxylabs AI Studio API key (AI Studio uses a single API key, no separate username/password required)
Register Oxylabs as a web provider Hermes can use for search and extraction:
hermes plugins install oxylabs/hermes-web-oxylabs --enableSet the API key in your current shell session:
export OXYLABS_API_KEY="your-api-key-here"Or add it directly to your Hermes environment configuration file (~/.hermes/.env):
OXYLABS_API_KEY=your-api-key-hereYou can set Oxylabs as the backend for search, extraction, or both.
hermes config set web.search_backend oxylabsFor queries requesting 10 or fewer results, the plugin uses AI Studio's /search/instant endpoint and returns results inline.
For larger result sets, it submits to /search/run to poll until completion.
web.search tool returns listings only (title, URL, description); full page content is fetched separately through the web.extract backend.
hermes config set web.extract_backend oxylabsEach URL is submitted to AI Studio's /scrape endpoint and polled until completed (up to 60 seconds per URL).
Batches of URLs are processed concurrently. A failure on one URL does not block the rest of the batch and returns an error field instead of content.
To use Oxylabs for both search and extraction, set the shared backend key via CLI:
hermes config set web.backend oxylabsOr update your ~/.hermes/config.yaml file:
web:
backend: "oxylabs"Backend and API key changes are applied at session start. Start a new Hermes session or restart the current one:
hermes restartRun a task that requires live web access or page extraction:
hermes run "Summarize the latest product updates from https://oxylabs.io"Success: The agent outputs structured search results or Markdown page content.
Error: If OXYLABS_API_KEY is not configured error is returned, the key wasn't picked up. Return to Step 2, then restart again.
Once Oxylabs is selected as backend, a missing key only returns an error instead of falling back to another provider.
The Hermes web_extract tool supports API base URL overriding (default: https://api-aistudio.oxylabs.io). You can override this endpoint for testing or self-hosted setup:
export OXYLABS_API_URL="https://your-custom-endpoint"Additional advanced features supported by Oxylabs AI Studio (format, render_javascript, geo_location) which are currently unsupported through Hermes dispatcher and may be available at a later date.
Integrating Oxylabs AI Studio with Hermes Agent bridges the gap between static AI agent reasoning and real-time web intelligence. Offloading web search and page extraction to Oxylabs helps your agents gain more reliable access to structured, LLM-ready context without the challenges of managing custom scraping infrastructure or vendor SDKs.
Want to explore further? Check out our guides on the best AI agent frameworks, MCP vs A2A protocols, and our LlamaIndex and CrewAI integrations for building web-powered LLM applications in Python.
If you have questions about web scraping and integrations, email us on support@oxylabs.io or use the live chat on our homepage.
Please be aware that this is a third-party tool not owned or controlled by Oxylabs. Each third-party provider is responsible for its own software and services. Consequently, Oxylabs will have no liability or responsibility to you regarding those services. Please carefully review the third party's policies and practices and/or conduct due diligence before accessing or using third-party services.
How to Navigate AI, Legal, and Web Scraping: Asking a Professional
In this interview, we sit down with a legal professional to shed light on the ever-changing legal framework surrounding web scraping.
Web Scraping With LangChain & Oxylabs API
Follow our quick guide on combining LangChain with Web Scraper API for hassle-free web scraping process and AI-driven analysis.
OpenAI Agents SDK Integration With Oxylabs Web Scraper API
Learn how to build AI agents that scrape and analyze web content by combining OpenAI's Agents SDK with Oxylabs Web Scraper API for cost-effective web access.
Get the latest news from data gathering world
Proxies
Advanced proxy solutions
Data Collection
Datasets
Resources
Innovation hub