In short. Content Signals let publishers declare search, AI-input and training preferences in robots.txt, with compliance dependent on crawler implementation.

Definition: Content Signals are website publishers' declared preferences about the use of their content for search, AI input and model training. The Content Signals policy expresses these preferences through a Content-Signal directive in robots.txt, the site's crawler instruction file.

Key facts

As of 17 October 2026, the policy text defines search, ai-input and ai-train; its search category excludes AI-generated search summaries.
Cloudflare's policy announcement explains that omitting a category conveys no permission or restriction through that signal.
Cloudflare's crawler configuration documentation distinguishes declared preferences from technical access restrictions and describes voluntary crawler compliance.
Google's robots.txt documentation lists user-agent, allow, disallow and sitemap as its supported fields.

What do the categories mean?

The policy specification defines search as building a search index and providing search results, including hyperlinks and short excerpts. AI-generated search summaries fall outside that definition. A search permission therefore expresses a narrower use than permission to supply content to an AI model.

The ai-input category covers feeding content into AI models. The separate ai-train category covers training or fine-tuning models.

For each category, yes permits collection for the corresponding use and no refuses it. An omitted category leaves that use without a permission decision through Content Signals. These values allow a publisher to express separate choices about indexing, answer production and training.

What does the file look like?

The policy site's Allow Search Only example contains these lines:

User-Agent: *
Content-Signal: ai-train=no, search=yes, ai-input=no
Allow: /

The asterisk addresses all user agents. Allow: / permits crawling throughout the site, while the content signal declares permitted and refused uses of the collected material. The policy provides human-readable explanatory text to accompany these directives as comments beginning with #.

Cloudflare's announcement releases the policy under a CC0 licence so others can implement it. Its example Content-Signal: search=yes, ai-train=no leaves ai-input unset. That omission differs from the explicit ai-input=no in the example above.

What are the adoption and enforcement limits?

Google's supported-field documentation places Content-Signal outside its documented robots.txt fields. Naming Googlebot in a policy example supplies instructions for that crawler; support still depends on the crawler operator's implementation.

Cloudflare's current documentation describes managed deployment through Bot Preference Sync and separately warns that robots.txt compliance is voluntary. It recommends AI Crawl Control for crawler-specific access blocks. A published preference and an enforced block perform different functions.

The same documentation describes an optional experimental content-use extension. Its managed example adds use=reference alongside search=yes, ai-train=no; reference means index, excerpt and link back. This extension has a distinct status from the policy's core categories.

FAQ

Can I allow search while refusing AI input?

Yes. The policy's search-only example combines search=yes with ai-input=no and ai-train=no. Whether a crawler follows those declarations depends on its implementation.

Does leaving out ai-input permit that use?

Cloudflare's policy explanation says an absent signal conveys no permission or restriction through Content Signals. You can express a refusal explicitly with ai-input=no.

How can I check a managed configuration?

Inspect the served /robots.txt response. Cloudflare's configuration documentation says it prepends managed content to an existing file and explains that policy comments alone express no specific preferences; look for the actual Content-Signal line.

Sources

Check how a page reads to answer engines. Open the GEO score analyzer