Are explicit index and follow directives necessary for Google?

Definition: Google Search uses index, follow as the default robots metadata behavior. An explicit directive with those values is optional; noindex and nofollow change page indexing and link following when Google can read the rule.

Key facts

• Google's meta tags documentation says that the default values are index, follow and do not need to be specified.
• nofollow tells Google not to follow links on a page. When the rule is absent, Google may use those links to find linked pages, according to Google's robots metadata reference.
• Google's robots metadata reference says that these settings can be read and followed only when crawlers can access the page that contains them.
• Google's noindex guide says that a crawled noindex rule removes the page from Google Search results, even when other sites link to it.

Does explicit index, follow add anything?

The answer is no when the goal is to retain Google's default behavior. The robots metadata reference was updated 24 March 2026, and Google's meta tags documentation states that index, follow is the default and does not need to be written. A page without a restrictive rule may be indexed, and Google may use its links to find linked pages.

An explicit form can be written as:

<meta name="robots" content="index, follow">

This tag restates the default. It does not create a separate permission that a page needs before Google can index it or use its links.

What changes the default?

noindex controls whether a page appears in Google Search. nofollow controls whether Google follows the links on that page. The Google robots metadata reference lists none as equivalent to noindex, nofollow, and says that a more restrictive rule applies when rules conflict.

The Google noindex guide describes noindex as a rule supplied either in a meta tag or an HTTP response header. When Googlebot crawls the page and extracts that rule, Google drops the page entirely from Google Search results, regardless of whether other sites link to it.

For a page that needs both restrictions, Google's reference shows this form:

<meta name="robots" content="noindex, nofollow">

Why does crawl access matter?

Google reads a robots meta tag or an X-Robots-Tag header during crawling. If a URL is disallowed in robots.txt, Google's robots metadata reference says that the indexing or serving information will not be found and will be ignored. A rule that must be followed therefore needs a page that Google can crawl.

The noindex guide gives the resulting failure mode: when a page is blocked by robots.txt or otherwise inaccessible, Googlebot never sees the noindex rule, and the page can still appear in search results if other pages link to it. The guide also says that the page must be crawled after the rule is added and that Googlebot may take months to revisit a page.

For non-HTML resources, the same guide documents the HTTP header form:

X-Robots-Tag: noindex

The guide says that a response header can serve non-HTML resources such as PDFs, video files, and image files.

Related terms

• Meta robots
• noindex
• nofollow
• robots.txt

FAQ

Should you add index, follow to every page?

No. Google's meta tags documentation gives index, follow as the default values, so an explicit tag is unnecessary when no restriction is required.

Does nofollow also keep a page out of Google Search?

Use noindex for page exclusion and nofollow for link-following control. Google's reference defines none as the combined noindex, nofollow rule.

What happens when robots.txt blocks a page with noindex?

Googlebot cannot read the page-level rule. Google's noindex guide says that the page can still appear in search results when other pages link to it.

Sources

1. Google Search Central, "Robots meta tag, data-nosnippet, and X-Robots-Tag specifications", 24 March 2026
2. Google Search Central, "Meta Tags and Attributes that Google Supports", 10 December 2025
3. Google Search Central, "Block Search Indexing with noindex", 10 December 2025

---

Check how a page reads to answer engines. Open the GEO score analyzer