In short. PerplexityBot is Perplexity’s search crawler, with a documented user-agent string and published IP ranges for verifying access.

Definition: PerplexityBot is Perplexity's automated web crawler for surfacing and linking websites in its search results. As of 17 October 2026, Perplexity's crawler documentation describes this search role and excludes crawling for AI foundation models from its purpose.

Key facts

Perplexity's crawler documentation recommends allowing PerplexityBot through both robots.txt and network security rules.
The same documentation distinguishes Perplexity-User, a separate agent that fetches pages in response to user actions and generally ignores robots.txt rules.
The Robots Exclusion Protocol defines Allow and Disallow rules for paths and requires the rules file at /robots.txt in the service's top-level path.

How does PerplexityBot identify itself?

A user-agent string identifies the client making a web request. Perplexity's published string is:

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)

The Robots Exclusion Protocol calls the crawler name used to select a rules group its product token. Here, that token is PerplexityBot; the full browser-like string belongs in request identification.

How does a website express its crawling preference?

A robots.txt file contains rules for automated clients. The protocol specification requires UTF-8 encoding and the text/plain media type. This example combines Perplexity's documented token with the protocol's permission syntax:

User-agent: PerplexityBot
Allow: /

To express a site-wide crawling restriction instead, the corresponding example is:

User-agent: PerplexityBot
Disallow: /

Under the protocol's matching rules, matching groups for the same token are combined, and the most specific matching path rule applies. Existing groups therefore need review before adding either example. The examples express crawling preferences; access to private content requires security controls, such as HTTP authentication, described in the specification's security section.

How can access be checked?

A web application firewall (WAF) filters incoming website requests. Perplexity's WAF instructions combine user-agent matching with source IP address verification. The linked PerplexityBot address file provides network prefixes for comparison.

An access review can therefore compare a request's identity and source address with the published configuration, then inspect whether the firewall allows it. Perplexity recommends refreshing address ranges periodically and monitoring logs after rule changes.

FAQ

Is PerplexityBot a model-training crawler?

Perplexity's documentation excludes crawling content for AI foundation models from its purpose.

Does a PerplexityBot rule cover user-requested visits?

Perplexity documents Perplexity-User separately. Its user-requested fetches generally ignore robots.txt rules.

How quickly do robots.txt changes take effect?

Perplexity's crawler guide states that its systems may take up to 24 hours to reflect changes.

Sources

Check how a page reads to answer engines. Open the GEO score analyzer