robots.txt Sitemap field

The Citead team, updated

In short. The robots.txt Sitemap field gives supporting crawlers the absolute URL of a sitemap or sitemap index, independently of user-agent groups.

Definition: The Sitemap field in robots.txt declares the full URL of a sitemap, a file listing website URLs for search engines. It can also point to a sitemap index, a file that lists other sitemaps.

Key facts

  • As of 17 October 2026, Google's robots.txt specification documents support by Google and Bing. The field requires an absolute URL, including its protocol and host.
  • The Sitemaps protocol allows a declaration to point to an index without separately listing every sitemap contained in it.
  • RFC 9309, section 2.2.4, permits crawlers to interpret sitemap records as an extension to the Robots Exclusion Protocol, the standard governing robots.txt crawl rules.
  • Google's sitemap submission guidance treats submission as a hint. Google decides whether to download the sitemap and use it for crawling.

What does the declaration look like?

Google's robots.txt specification gives these sitemap locations in its example:

sitemap: https://example.com/sitemap.xml
sitemap: https://cdn.example.org/other-sitemap.xml

Each line contains a field name, a colon and the sitemap URL. The field name is case-insensitive, so Sitemap and sitemap are equivalent. The URL value is case-sensitive. A relative path such as /sitemap.xml lacks the required protocol and host.

The same documentation allows multiple declarations and a sitemap hosted on a different host from robots.txt. This makes the second line above a supported form of location declaration.

Does the field belong to a user-agent group?

A user-agent group associates a crawler name with crawl rules. The Sitemaps protocol's robots.txt instructions make the sitemap declaration independent of the User-agent line: its position in the file does not restrict it to the adjacent crawler group.

RFC 9309 makes interpreting these additional records optional. It also requires their parsing to leave the standard records intact, specifically stating that a sitemap record must not terminate a group. Support therefore depends on the crawler's implementation; compliance with the robots.txt standard alone leaves sitemap handling optional.

How can a website check its setup?

Google's build and submit a sitemap guide says it finds a robots.txt declaration the next time it crawls that file. Checking the published file establishes which sitemap URL it exposes.

For sitemaps hosted elsewhere, the same guide describes creating a separate sitemap for each site and referencing that site's sitemap from its own robots.txt. Its example places this declaration at https://example.com/robots.txt:

sitemap: https://sitemaps.example.com/sitemap-example-com.xml

The guide also offers submission through Search Console's Sitemaps report. That report shows when Googlebot accesses the sitemap and reports potential processing errors, providing a check beyond reading the declaration itself.

FAQ

Can I declare several sitemap files?

Yes. The Sitemaps protocol allows multiple sitemap declarations in robots.txt. You can also reference a sitemap index that lists the individual files.

Do I need a Search Console submission as well?

Google's submission guide presents robots.txt and Search Console as ways to make a sitemap available. You can use the Sitemaps report for its access and processing information.

Sources

  1. Google, "How Google interprets the robots.txt specification", 31 August 2026
  2. Sitemaps.org, "Sitemaps XML format", 21 November 2016
  3. IETF, "RFC 9309: Robots Exclusion Protocol", September 2022
  4. Google Search Central, "Build and submit a sitemap", 8 July 2026

Check how a page reads to answer engines. Open the GEO score analyzer

Test a page and see what to fix first

The analysis usually takes less than 10 seconds and needs no account.

400,000+ pages scored in the public benchmark