Does Brave publish a separate search crawler user agent?

Definition: A user agent string is a request-header value describing the client software making a web request. No: as of 17 October 2026, Brave says its search crawler does not advertise a differentiated user agent.

Key facts

• Brave's Search API FAQ attributes its data to its own web crawler and signals from users who opt in to the Web Discovery Project.
• Brave's crawler help page says its bot avoids crawling a domain or page that Googlebot cannot crawl.
• The IETF's HTTP Semantics specification describes the User-Agent value as product identifiers with optional comments.

What does Brave's identification policy say?

Brave's Brave Search Crawler documentation gives its reason: some websites allow only Google to crawl them. The page leaves the exact request-header value unspecified.

The reference to Googlebot concerns crawl access. In the Search API FAQ, Brave separately describes its results as coming from an independent index that Brave crawls and ranks. Reading these statements together, Googlebot accessibility is a condition in Brave's crawling policy; Brave operates the index supplying its API results.

For interpreting request headers, the HTTP Semantics specification says each product identifier contains a name and an optional version. Its explanation covers how software describes itself in a request. A header-based filter for Brave would require a distinct identifier that Brave's published policy explicitly declines to advertise.

What controls does Brave document for website owners?

Brave's crawler documentation distinguishes crawl access from index removal. It directs site owners to the noindex directive for delisting. This instruction asks supporting search engines to exclude content from their index.

For an HTML page, Google's noindex implementation guide gives this tag for search engines that support the rule, placed inside the page's <head> element:

<meta name="robots" content="noindex">

The robots name addresses supporting search engines generally. Google's guide says noindex can also be delivered through a response header. For the instruction to be effective, the page must remain accessible to the crawler. Blocking access through robots.txt, the file carrying crawler access rules, prevents the crawler from seeing the tag.

Brave says it needs to fetch the updated page again to apply the directive. Its documentation links to the URL submission form for requesting another fetch.

Related terms

• Brave Search API
• robots.txt
• noindex
• User-agent spoofing

FAQ

Can I filter Brave requests by a published, separate user agent?

Brave's crawler policy supplies no separate Brave header value for such a filter.

How can I request removal of a page from Brave Search?

Brave directs you to add noindex, then let its crawler fetch the updated page.

Sources

1. Brave, "Brave Search API", accessed 17 October 2026
2. Brave, "Brave Search Crawler", accessed 17 October 2026
3. IETF, "HTTP Semantics", June 2022
4. Google Search Central, "Block Search indexing with noindex", accessed 17 October 2026

---

Check how a page reads to answer engines. Open the GEO score analyzer