In short. Meta robots gives search engine crawlers page-level indexing and preview rules that Google reads when it can crawl the page.

Definition: Meta robots is an HTML meta tag that supplies page-level instructions to search engine crawlers about indexing and presentation in search results. Its name attribute identifies the intended crawler audience, and its content attribute contains the rules.

Key facts

As of 17 October 2026, Google's robots meta tag specification recommends placing the tag in the HTML <head> and treats its name and content values as case-insensitive.
Google's robots.txt introduction explains that a URL blocked from crawling can still appear in search results when other pages link to it.
Google's AI features guide requires a page to be indexed and eligible for a Search snippet to qualify as a supporting link in AI Overviews or AI Mode, Google's AI search features.

How is the tag written?

Google's specification gives this instruction for excluding an HTML page from search results:

<meta name="robots" content="noindex">

The tag belongs in the page's <head>. A content management system may expose a search engine settings screen for adding it without editing HTML directly.

The value robots addresses search engine crawlers generally. A crawler-specific tag uses that crawler's user agent token, the identifier used to address it. Google supports googlebot for text results and googlebot-news for news results. Its example for suppressing Google's search snippet, the text preview accompanying a result, is:

<meta name="googlebot" content="nosnippet">

The specification notes that other search engines may interpret rules differently.

What do the main rules control?

Google's rule reference defines noindex as excluding the page from search results. The nofollow value instructs crawlers not to follow links on the page. The value none combines noindex and nofollow.

The nosnippet value suppresses text snippets and video previews; a static image thumbnail may remain. The max-snippet value limits a text snippet by character count. Google's example allows up to 20 characters:

<meta name="robots" content="max-snippet:20">

A value of max-snippet:0 is equivalent to nosnippet. Google also documents image preview size controls through max-image-preview and video preview duration controls through max-video-preview.

How do crawl access and combined rules interact?

Google's specification states that crawlers find meta robots instructions when fetching the URL. A robots.txt crawl block prevents Google from reading those instructions. Crawl access must therefore remain available for a page's noindex instruction to take effect.

Multiple rules can share a comma-separated content value or appear in separate tags. Google combines applicable restrictions, including those in general and crawler-specific tags. Its example is:

<meta name="robots" content="nofollow">
<meta name="googlebot" content="noindex">

Googlebot interprets this pair as noindex, nofollow. For conflicting preview rules, the more restrictive instruction applies: nosnippet takes precedence over max-snippet:50.

What changes for Google's AI search, and how is it checked?

Google's robots specification states that nosnippet prevents page content from serving as direct input to AI Overviews and AI Mode. The max-snippet setting limits how much content can serve as direct input, subject to separately permitted uses such as content supplied through structured data or a licence agreement.

Google's AI features guide recommends the URL Inspection tool to examine the HTML Googlebot receives and verify that preview controls are visible. Changes require recrawling and processing; the guide gives a range from several days to several months.

FAQ

Can I apply meta robots to a PDF?

For a PDF, you can send an X-Robots-Tag HTTP response header. Google's specification documents this mechanism for non-HTML resources.

Must I add an explicit rule to allow indexing?

Google's rule reference defines all as the default, with no indexing or serving restrictions. Explicitly listing it has no effect.

Can I suppress a preview for just one passage?

Google's specification provides the data-nosnippet HTML attribute for span, div, and section elements. You can use it to exclude selected text from snippets.

Sources

Check how a page reads to answer engines. Open the GEO score analyzer