Can several X-Robots-Tag headers be returned for one resource?

Definition: Yes. Google documents that a response may contain multiple X-Robots-Tag headers, and that the rules may also be written as a comma-separated list. The headers carry crawler instructions for the URL's response, including resources that do not use HTML.

Key facts

• Google's robots metadata reference says that X-Robots-Tag is an HTTP response-header element for a given URL.
• Google says that rules available in a robots meta tag can also be specified as an X-Robots-Tag.
• Google documents two equivalent forms for combining rules: multiple X-Robots-Tag header lines or a comma-separated list of rules.
• RFC 9110, the IETF specification for HTTP Semantics, says that repeated field names are combined in received order with each field-line value separated by a comma.

What does Google document about several headers?

Google describes X-Robots-Tag as part of the HTTP response for a URL. It gives the same rule vocabulary as the robots meta tag and identifies it as the response-header option for non-HTML resources such as PDF files, video files, and image files. The documentation says that multiple headers can be combined within the response, or that a comma-separated list of rules can be used.

Its example places two directives on separate header lines:

X-Robots-Tag: noimageindex
X-Robots-Tag: unavailable_after: 25 Jun 2010 15:00:00 PST

The same two directives can be expressed on one line as a list:

X-Robots-Tag: noimageindex, unavailable_after: 25 Jun 2010 15:00:00 PST

Google also documents an optional user-agent name before the rules. A rule without a user-agent name applies to all crawlers, and the HTTP header, user-agent name, and values are case-insensitive.

How does HTTP combine repeated field lines?

RFC 9110 defines a field line as a field name together with the value for that instance. When a field name occurs once, its field value is that line's value. When the same name occurs repeatedly, the combined value is the corresponding list in order, with each value separated by a comma. The RFC illustrates the rule with two Example-Field lines whose combined value is Foo, Bar, Baz.

The RFC also says that a recipient may combine repeated field lines into one line by appending later values in order, separated by a comma and optional whitespace. The order matters when interpreting the field value, so a proxy must preserve the order of same-name field lines when forwarding the message. A sender must not generate repeated field lines unless the field definition allows values to be recombined as a comma-separated list. Google's explicit X-Robots-Tag guidance supplies that permission for this directive format.

What should a site owner check?

Inspect the complete HTTP response for the resource and record whether the server sends separate lines, one comma-separated line, or both. Compare the directives after the HTTP field-combination step, because the two separate lines in Google's example form one ordered list when combined. Check the rule scope as well: Google documents that an optional user-agent name can precede rules, while rules without one apply to all crawlers.

Google says that robots meta tags and X-Robots-Tag headers are discovered when a URL is crawled. If robots.txt disallows crawling, information about the indexing or serving rules will not be found and will be ignored.

Related terms

• X-Robots-Tag
• Meta robots
• noindex
• nofollow

FAQ

Can one X-Robots-Tag line contain several directives?

Yes. Google documents a comma-separated list of rules, so one line can carry several directives. The separate-line form is also documented.

Does the order of repeated headers matter?

RFC 9110 says that same-name field lines are combined in the order received and that a proxy must preserve that order when forwarding the message.

Can a header target one crawler?

Yes. Google's documentation says that an X-Robots-Tag may specify a user-agent before its rules. A rule without a user-agent name applies to all crawlers.

Sources

1. Google Search Central, "Robots meta tag, data-nosnippet, and X-Robots-Tag specifications", updated 24 March 2026
2. IETF, "RFC 9110: HTTP Semantics", June 2022

---

Check how a page reads to answer engines. Open the GEO score analyzer