What robots directives does Yoast add to internal search pages?

Definition: Yoast SEO gives internal search results pages, which list matches for searches within a website, the robots directives noindex, follow by default. These instructions exclude the results page from indexing by supporting search engines while allowing its links to be followed.

Key facts

• As of October 2026, Yoast's noindex help documentation identifies noindex, follow as the default for search pages.
• Yoast's meta robots specification places the robots tag in the HTML <head> and gives noindex, follow as the output for pages excluded from search results.
• Google's noindex documentation requires the page to remain accessible to its crawler so that the directive can be read.

What HTML should a search page contain?

A meta robots tag is an HTML element that supplies page-level instructions to search engines. Yoast's functional specification gives this example for a page treated as non-public, meaning a page excluded from search results:

<meta name="robots" content="noindex, follow" />

The two values address separate actions: indexing the results page and following links from it. Yoast's WordPress noindex guide applies this combination specifically to internal search results and explains that links on those pages should remain followed.

The specification combines applicable robots values and resolves conflicts in favour of the more restrictive instruction. In particular, noindex takes precedence over index, and nofollow takes precedence over follow. The example above therefore describes the documented default; additional restrictions can change the final combination.

Is a setting required for internal search results?

Yoast's WordPress noindex guide describes internal search exclusion as an automatic feature and says it has no editable setting. The same guide identifies developer filters as a way to change this behaviour.

Yoast also documents a separate option called Prevent crawling of internal site search URLs in its crawl optimization settings. Enabling that option adds a disallow rule to robots.txt, the file used to control crawler access to URL paths. Yoast generally recommends allowing crawling so that search engines can read the automatic noindex tag, and identifies excessive crawling or attacks on search pages as reasons to enable the blocking option.

How can the directive be checked?

The relevant inspection target is the search results URL. In Google's noindex documentation, the HTML implementation places the robots tag in the <head>. Checking that element establishes which directives the page supplies.

For Google's view of the page, the same documentation recommends Search Console's URL Inspection tool to examine the HTML received by Googlebot, Google's crawler. Its Page Indexing report also tracks pages from which Googlebot extracts noindex. If a results URL remains in Google Search, the documentation identifies a pending recrawl or a robots.txt block as possible causes.

Related terms

• Meta robots
• noindex
• robots.txt
• Can Google read noindex on a page blocked by robots.txt?

FAQ

Does the default also prevent links from being followed?

Yoast's internal search explanation specifies noindex, follow. The follow value allows links on the results page to be followed.

Should I enable the crawl-blocking option to apply noindex?

Yoast's crawl optimization documentation says the plugin already adds noindex automatically. Its crawl-blocking option separately adds a robots.txt disallow rule.

Why might a search page still appear in Google?

Google's debugging guidance says Googlebot must crawl the page to see the rule. You can use URL Inspection to check the received HTML and request a recrawl.

Sources

1. Yoast, "Yoast SEO Meta robots: Functional specification", undated
2. Yoast, "How to noindex URLs", updated 29 September 2026
3. Yoast, "Noindex a post in WordPress, the easy way!", updated 17 December 2025
4. Yoast, "Yoast SEO settings: Crawl optimization", updated 7 April 2026
5. Google Search Central, "Block Search indexing with noindex", updated 10 December 2025

---

Check how a page reads to answer engines. Open the GEO score analyzer