Claude web search

The Citead team, updated

In short. Claude web search retrieves current web information for cited answers, with separate crawler controls for website owners.

Definition: Claude web search is a feature of Anthropic's Claude AI assistant that retrieves current web information for conversational answers with source citations. As of 17 October 2026, Anthropic documents search in Claude and a web search tool for applications using its API, an interface through which software sends requests to Claude.

Key facts

  • Anthropic's web search announcement records global availability across all Claude plans in its update of 27 May 2025.
  • The Claude help guide describes responses that combine information from multiple sources.
  • Anthropic's crawler documentation identifies separate robots for search, user-directed retrieval and potential model training.
  • The API documentation provides controls for limiting searches and filtering source domains.

How does Claude use web information?

According to Anthropic's web search announcement, Claude incorporates web information into conversational responses and links directly to sources. Those citations give readers a route to the material behind an answer. Anthropic gives product comparisons involving features, prices and reviews as an example of a web search use case.

The help guide also describes web fetch: retrieval of a page's content when a person supplies its URL. This lets Claude analyze a particular article or other web page. Search can draw on multiple sources, while a supplied URL directs retrieval toward particular content.

Which controls matter for a website?

An automated web client is commonly called a bot or crawler. Anthropic's crawler documentation assigns Claude-SearchBot to improving search results and Claude-User to requests initiated by people using Claude. ClaudeBot collects material that could contribute to model training. These roles give a website owner separate access choices for search and training.

A robots.txt file contains instructions for crawlers. Anthropic states that its bots honor these directives. Applying its documented blocking pattern to the search bot gives this rule:

User-agent: Claude-SearchBot
Disallow: /

This is a blocking rule for the whole site. Anthropic instructs owners to apply exclusions separately to each relevant subdomain. The documented consequence of blocking the search bot is exclusion from Anthropic's content indexing for search optimization; blocking Claude-User prevents retrieval in response to user queries. Reviewing these permissions is therefore relevant when assessing a site's access settings for Claude.

How does search work in applications?

Anthropic's API guide describes a search cycle in which Claude decides when a search is useful, the API executes it, and Claude receives the results. The cycle can repeat within a request before Claude returns an answer with citations.

Application developers can use max_uses to cap searches per request. Domain filters select permitted domains through allowed_domains or exclude domains through blocked_domains. These settings affect the pool of sources an application searches. The guide states that citations are always enabled for the web search tool.

FAQ

Anthropic's help guide describes a Web search option under the plus button in the chat window. It also says that the new Claude experience searches when useful without a separate search toggle, so the control you see depends on the interface.

Can you keep training access separate from search access?

Yes. Anthropic's crawler documentation gives ClaudeBot, Claude-SearchBot and Claude-User distinct roles. You can set robots.txt instructions for each bot according to the access you want to permit.

Sources

  1. Anthropic, "Claude web search now available globally on all plans", update 27 May 2025
  2. Anthropic, "Enable and use web search", accessed 17 October 2026
  3. Anthropic, "Does Anthropic crawl data from the web, and how can site owners block the crawler?", accessed 17 October 2026
  4. Anthropic, "Web search tool", accessed 17 October 2026

Check how a page reads to answer engines. Open the GEO score analyzer

Test a page and see what to fix first

The analysis usually takes less than 10 seconds and needs no account.

400,000+ pages scored in the public benchmark