In short. Bot crawl frequency counts crawler requests within a defined period and scope, while citations require separate observation of answer links.

Definition: Bot crawl frequency is the number of requests attributed to a web crawler during a defined time period, for a specified website or set of URLs. It measures fetching activity, including repeated requests for the same resource.

Key facts

As of 17 October 2026, the documentation provides these measurement details:

Google's Crawl Stats documentation counts duplicate requests individually and includes unsuccessful requests in its totals.
Apache HTTP Server's access log documentation defines a request timestamp and a Combined Log Format that records the User-Agent request header.
Google's crawler verification documentation provides reverse and forward DNS checks, or matching against published IP ranges, to verify requests attributed to Google.

What does Google Crawl Stats count?

Google's Crawl Stats report includes requests for page resources hosted within the selected site's scope. A total therefore includes more than HTML page fetches. Each request in a server-side redirect chain counts separately, within the report's domain scope.

The report groups requests by response, file type, crawl purpose and Googlebot type. Selecting HTML narrows the view to HTML responses; selecting Refresh isolates recrawls of known pages. These views help distinguish repeated page fetching from resource requests.

Some totals include potential fetches abandoned because robots.txt is unavailable. Those requests never reach the server, which explains one documented difference between Crawl Stats and server logs. Record the property scope and selected grouping alongside any total used for comparison.

How is frequency measured in access logs?

An access log records requests received by a web server. Apache's Combined Log Format supplies fields for the requesting address, timestamp, request line, response status and reported User-Agent:

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" combined
CustomLog "log/access_log" combined

In this format, %t records when the request arrives, including a time zone. %r records the request line, and %>s records the response status. For measurement, group matching requests into fixed time intervals and count each request in its interval. An average rate is the request count divided by the duration, expressed in a stated unit such as requests per hour.

Keep the time zone, URL scope and bot identification method consistent between periods. Report distinct requested URLs separately when measuring coverage: repeated requests increase frequency without increasing the number of distinct URLs.

The User-Agent field contains the client's reported identity. Google's verification procedure checks the requesting IP address through reverse DNS, verifies the returned domain and resolves that name back to the original IP address. Apply that verification before labelling requests as verified Google crawler traffic.

What can a change in frequency show?

Google's Crawl Stats troubleshooting guidance identifies slow responses and increased server errors as reasons Googlebot reduces requests. It also lists adding a new site section or removing crawl restrictions as possible causes of increased crawling.

Compare request counts with response times and error responses over the same interval. A change in volume becomes useful when its associated crawler type, resources and response outcomes are identified.

Does frequency measure citations in AI answers?

A request count alone cannot establish whether an answer includes a source link to the fetched page. Google's AI features documentation specifies that a page eligible as a supporting link in AI Overviews or AI Mode must be indexed and eligible for a Search snippet.

The same documentation places traffic from these features in Search Console's Performance report under the Web search type. Keep crawl counts as a fetching measure and record observed answer links separately when measuring citations.

FAQ

Where do I open Google's Crawl Stats report?

In Search Console, open Property settings, then Crawl stats. Google's report documentation specifies that access requires a root-level property, either a Domain property or a root-level URL-prefix property.

What if a URL is missing from the report's examples?

Google describes the example URLs as a representative sample in its Crawl Stats documentation. A missing example URL can still have received a crawl request.

Sources

Check how a page reads to answer engines. Open the GEO score analyzer