In short. OAI-SearchBot is OpenAI's search crawler, with robots.txt access controls independent of GPTBot's training controls.

Definition: As of 17 October 2026, OpenAI documents OAI-SearchBot as its web crawler for surfacing websites in ChatGPT search results. Website owners control its access through rules in robots.txt, a file that tells crawlers which paths they may access.

Key facts

OpenAI's crawler documentation assigns OAI-SearchBot to search and GPTBot to collecting content that may be used for model training.
OpenAI's independent crawler settings allow a website to permit search crawling while declining training crawling.
OpenAI's update guidance says search systems can take about 24 hours to adjust after a robots.txt change.
The Robots Exclusion Protocol, published as RFC 9309 in September 2022, places robots.txt at the service's top-level path.

How does a website allow search crawling separately from training?

OpenAI's crawler documentation describes separate OAI-SearchBot and GPTBot settings. The following example expresses that separation using the user-agent groups and path rules defined in the robots.txt specification:

User-agent: OAI-SearchBot
Allow: /

User-agent: GPTBot
Disallow: /

The OAI-SearchBot group permits paths beginning with /; the GPTBot group disallows them. This is a configuration example assembled from the documented controls. The standard's access method requires the file at /robots.txt, encoded as UTF-8 and served as text/plain.

How are existing robots.txt rules applied?

The standard's user-agent rules require case-insensitive matching of crawler names. When a matching named group exists, its rules apply. The wildcard group, identified by User-agent: *, applies when no named group matches. Consequently, adding an OAI-SearchBot group requires checking any path restrictions previously expressed only in the wildcard group.

When several groups match the same crawler, the standard requires combining their rules. Within those rules, the path-matching specification selects the most specific matching path, measured by its length in octets. A named group can therefore allow the root path while restricting longer path prefixes with disallow rules. Path matching should be case sensitive; an equivalent allow rule should take precedence over a disallow rule.

OpenAI also recommends permitting requests from its published IP ranges in its crawler documentation, which links to https://openai.com/searchbot.json.

FAQ

Can you allow OAI-SearchBot while blocking GPTBot?

Yes. OpenAI's crawler settings treat search and training access independently, as the configuration above illustrates.

Does blocking OAI-SearchBot remove every link to your site?

OpenAI's crawler documentation says opted-out sites are excluded from ChatGPT search answers but can still appear as navigational links.

Sources

Check how a page reads to answer engines. Open the GEO score analyzer