In short. Google must crawl a page to read its noindex instruction and exclude it from Search results.

Definition: As of October 2026, Google's robots meta tag specification defines noindex as a rule that excludes a page, media item or resource from search results. The crawler must be able to access the resource to read and follow the rule.

Key facts

Google's robots meta tag specification supports noindex in an HTML meta tag or an X-Robots-Tag HTTP response header.
Google's index exclusion guide says both implementations have the same effect and that Google does not support noindex in robots.txt.
Google's robots.txt introduction explains that a blocked page's URL can still appear in results when other pages link to it.
Google's AI features documentation requires a page to be indexed and eligible for a search snippet to appear as a supporting link in AI Overviews or AI Mode.

How is noindex implemented?

Google's robots meta tag specification places the HTML instruction in the page's <head> section:

<meta name="robots" content="noindex">

The robots name addresses search engine crawlers. For Google-specific targeting, the name can be googlebot. The name and content values are case-insensitive.

An HTTP response header carries the instruction for resources such as PDFs, which cannot contain an HTML meta tag:

X-Robots-Tag: noindex

The same specification allows robots meta tag rules in this header. Configuration therefore depends on the content type and on whether the website can edit its HTML or response headers.

Why must crawling remain possible?

Crawling is the crawler's retrieval of a URL. Google's robots.txt introduction defines robots.txt as the file that controls which URLs crawlers can access. A blocked web page can still have its URL indexed through links from elsewhere, with no description in the search result.

Google's index exclusion guide requires a page carrying noindex to remain accessible to Googlebot, Google's search crawler. Blocking that URL in robots.txt prevents Googlebot from reading the instruction. Once Googlebot fetches the page and extracts noindex, Google removes it from Search results even if other sites link to it.

What changes for Google's AI search features?

Google's AI features documentation requires indexing and snippet eligibility for supporting links in AI Overviews and AI Mode. A page excluded from the index by noindex therefore fails that eligibility requirement.

The same documentation identifies Googlebot's robots.txt rules as the crawl access control for Search and lists noindex among the controls for limiting information shown in Search. It treats controls for training and grounding in some other Google systems separately, through Google-Extended.

How is the implementation checked?

Google's index exclusion guide recommends the URL Inspection tool to examine the HTML Googlebot received. Its Page Indexing report in Search Console tracks pages where Googlebot extracted a noindex rule.

A page can remain in results until Google crawls it again. The same guide says a revisit may take months and allows a recrawl request through URL Inspection. Verification therefore includes both the delivered instruction and Google's subsequent crawl.

FAQ

Can I put noindex in robots.txt?

Google's index exclusion guide says this is unsupported. Use the HTML meta tag or the HTTP response header, and keep the URL accessible to Googlebot.

Does noindex make a page private?

The rule controls search indexing. Google's robots.txt introduction recommends password protection for private files that need protection from web crawlers.

Does every search engine interpret noindex identically?

Google's index exclusion guide warns that other search engines may interpret the rule differently. Its documented removal behavior applies to Google Search.

Sources

Check how a page reads to answer engines. Open the GEO score analyzer