In short. Crawl Stats separates requests by Googlebot type, with linked-resource fetches grouped under Page resource load.
Definition: Googlebot type is the Crawl Stats grouping for the user agent making a crawl request. Each group has its own request total.
Key facts
- As of 17 October 2026, Google's Crawl Stats documentation lists Smartphone, Desktop, Image, Video, Page resource load, AdsBot, StoreBot and Other agent type.
- Google's report announcement describes separate breakdowns by response code, file type, crawl purpose and Googlebot type.
- Google's Googlebot documentation defines Smartphone as simulating a mobile user and Desktop as simulating a desktop user.
Where are the crawler groups and request totals?
Google's Crawl Stats help gives this report path:
Search Console > Property settings > Crawl stats
The report's launch announcement describes totals grouped by Googlebot type. Clicking a grouping-table row opens example URLs for that type. The overview also charts total requests, total download size and average response time over time. These are separate measurements: the request count measures fetch activity, download size measures bytes, and response time measures how long responses take.
What does each group include?
Google's crawler-type definitions describe these groups:
| Group | Meaning |
|---|---|
| Smartphone | Googlebot smartphone. |
| Desktop | Googlebot desktop. |
| Image | Googlebot image requests. |
| Video | Googlebot video requests. |
| Page resource load | Secondary fetches of linked resources needed to render a page, including images and CSS. |
| AdsBot | Requests from AdsBot crawlers. |
| StoreBot | Requests from the product shopping crawler. |
| Other agent type | Requests from another Google crawler. |
Images and videos fetched as page resources count under Page resource load. This applies even though Image and Video have their own crawler groups.
How should the totals be read?
Google's total-request definition includes successful and unsuccessful requests. Repeated requests to the same URL count individually. On-site resources count; externally hosted resources fall outside the total. Potential fetches abandoned because robots.txt is unavailable also count, with limited grouping details possible.
Google's Googlebot guide adds context to the Smartphone and Desktop split. Most sites are primarily indexed using their mobile content, so the mobile crawler accounts for most Googlebot requests on those sites. The guide also explains that rendering fetches referenced resources separately, including CSS and JavaScript.
Related terms
- Googlebot
- How do I see which file types Google crawls on a site?
- How do I compare discovery and refresh crawls in Search Console?
- Server log analysis
FAQ
Can I use the example URLs as a complete request list?
Google's Crawl Stats documentation describes them as a representative sample. A URL can receive requests without appearing among the examples.
Can I block Smartphone while allowing Desktop?
Google's Googlebot documentation says both obey the same robots.txt product token. You can distinguish their subtypes through the HTTP user-agent request header, but robots.txt rules cannot target either subtype separately.
Sources
- Google Search Console Help, "Crawl Stats report", undated
- Google Search Central Blog, "New and improved crawl stats for your site", 24 November 2020
- Google Search Central, "Googlebot", updated 3 February 2026
Check how a page reads to answer engines. Open the GEO score analyzer