In short. Google-Extended controls specified Gemini training and grounding uses through robots.txt while leaving Google Search inclusion and ranking unaffected.
Definition: As of 17 October 2026, Google-Extended is Google's robots.txt product token for controlling whether crawled website content may support future Gemini model training and specified Gemini grounding uses. Grounding supplies content from the Google Search index to a model at prompt time to improve factuality and relevance.
Key facts
- Google's crawler documentation identifies
Google-Extendedas the exact token used in robots.txt. - Google documents no separate HTTP user-agent string for Google-Extended; crawling uses existing Google user-agent strings.
- The documented scope includes future Gemini model training and grounding in Gemini Apps and Grounding with Google Search on Vertex AI.
- Google states that Google-Extended leaves Search inclusion and ranking unaffected.
Which uses does Google-Extended control?
Google's product description covers training future generations of Gemini models powering Gemini Apps and Vertex AI API for Gemini. It also covers supplying Search index content at prompt time in Gemini Apps and Grounding with Google Search on Vertex AI. A publisher's decision therefore covers both training and these answer-time uses.
For AI Overviews and AI Mode in Google Search, Google's AI features documentation identifies Googlebot robots.txt directives as the crawl-access control. To limit information displayed from pages in Search, it lists nosnippet, data-nosnippet, max-snippet and noindex. These Search controls serve a different scope from the Google-Extended product token.
How is a site-wide restriction written?
A robots.txt file is a text file containing rules grouped by user agent. The following example combines Google's Google-Extended token with the site-wide Disallow: / syntax in its robots.txt instructions:
User-agent: Google-Extended
Disallow: /
This group restricts the covered uses across the site. Google's file-location rules require robots.txt at the root of the host, with UTF-8 text encoding. Its rules apply only to the protocol, host and port where the file is posted. A restriction on a main domain therefore needs separate attention for subdomains.
How can the configuration be checked?
Google's testing instructions recommend opening the published robots.txt address in a private browsing window to check public access. The robots.txt report in Search Console helps identify markup issues in an already accessible file. Google's open source robots.txt library also supports local testing.
Server logs cannot identify Google-Extended by a dedicated HTTP user-agent string: Google's HTTP user-agent documentation says crawling uses existing Google user-agent strings.
Related terms
FAQ
Does blocking Google-Extended remove my site from Google Search?
Google's documentation says Google-Extended has no effect on Search inclusion and is not used as a Search ranking signal. Googlebot access remains a separate configuration decision.
Can I use Google-Extended only to refuse model training?
The documented token covers both future Gemini training and the specified grounding uses. The same Google-Extended rule applies to that combined scope.
Must I submit the robots.txt file to Google?
Google's submission instructions say its crawlers automatically find and use an uploaded, tested robots.txt file. You can request a refresh of Google's cached copy after an update through the linked update procedure.
Sources
- Google, "Google's common crawlers", 15 October 2026
- Google Search Central, "AI features and your website", 10 December 2025
- Google, "How to write and submit a robots.txt file", 21 November 2025
Check how a page reads to answer engines. Open the GEO score analyzer