Optimisation, , 8 min read

llms.txt: what the file is for and how to write it

llms.txt promises to guide AI through your site. Google does not use it and 97% of files are never read. Who it serves, and how to write one.

Citead TeamGEO Experts

Key points

  • The specification only requires an H1 title with the site name.
  • Google calls llms.txt unnecessary for its Search, including AI Overviews and AI Mode.
  • 97% of llms.txt files received no requests in May 2026 (Ahrefs).
  • Coding agents are its main readers today.

Summary: llms.txt is a Markdown file placed at the root of a website to summarize its content for language models, proposed by Jeremy Howard (Answer.AI) in September 2024. Google states that its Search, including AI Overviews and AI Mode, does not use it. Across 137,000 domains studied by Ahrefs, 97% of llms.txt files received no requests in May 2026. Its main readers are coding agents working with technical documentation.

What is an llms.txt file?

An llms.txt file is a Markdown summary of a website, published at /llms.txt, that tells a language model what the site contains and where to find the useful pages. Jeremy Howard, co-founder of Answer.AI, proposed it on September 3, 2024; version 2 of the proposal, updated in August 2026, reflects two years of adoption.

The motivation is practical. A web page mixes its content with navigation, scripts and interface elements; an agent reading it has to convert everything to text and spends context for nothing. The file offers a short, structured entry point. The proposal also recommends publishing a Markdown version of important pages at the same address with a .md suffix.

llms.txt replaces neither robots.txt nor sitemap.xml. The former tells crawlers what they may access, the latter lists every indexable page; llms.txt offers an annotated selection, read when an agent needs information.

How do you write an llms.txt file?

The specification sets a precise order:

  1. an H1 title with the name of the site or project, the only required element;
  2. a blockquote (>) summarizing the site in a few sentences;
  3. paragraphs or lists, without headings, giving useful context;
  4. H2 sections containing lists of links, each link optionally followed by : and a note;
  5. by convention, an ## Optional section for links an agent can skip when context is short.

Here is the top of the file published by citead.com, available at citead.com/llms.txt:

# Citead

> Citead is a Generative Engine Optimization (GEO) platform. It scores any web page from 0 to 100 on how easily AI assistants such as ChatGPT, Perplexity, Claude and Google AI Overviews can understand, quote and cite it, shows the criteria behind the score and lists the fixes to apply. A higher score makes a page easier to cite; it does not predict or guarantee a citation. Paid plans also check every week whether a site is actually cited by ChatGPT, Gemini, Perplexity and Google AI Overviews.

[...]

## Free tools

- [GEO score tool](https://citead.com/geo-score-tool): Paste a URL, get its GEO score, the score of each criterion and the fixes to apply, no account. Rule-based and deterministic. French: https://citead.com/outil-score-geo
- [GEO editor](https://citead.com/geo-score-analyzer): Write or paste a text, the GEO score updates live with corrections. French: https://citead.com/analyseur-score-geo
- [llms.txt generator](https://citead.com/llms-txt-generator): Enter a domain, get a spec-compliant llms.txt and llms-full.txt built from the real pages, plus an audit of the existing file. French: https://citead.com/generateur-llms-txt

The file goes at the root, or in a subdirectory (/docs/llms.txt) to cover only the pages under that path. Version 2 recommends pointing to it from pages with the rel="describedby" link relation, and pointing to a page's Markdown version with rel="alternate" type="text/markdown". Our llms.txt generator builds a compliant file from a domain's real pages and audits the existing one.

Does Google use llms.txt?

No. Google Search Central's AI optimization guide puts it on a list of practices to ignore: there is no need to create machine-readable files or AI text files to appear in Google Search, including its generative AI features, because Google Search does not use them. The same guide notes that you can publish them for other services that use them, with no effect on visibility in Google.

Google's John Mueller has taken the same line publicly. Asked whether llms.txt files on Google's own sites meant an endorsement, he answered no (Search Engine Roundtable). He also called its value "purely speculative", noting that the file has existed for years without any AI system using it (Search Engine Journal).

The signal from Google is not uniform: the Chrome team added an llms.txt check to Lighthouse's experimental agentic browsing audits, arguing that without the file agents may spend more time exploring a site. Asked about the contradiction, John Mueller explained that llms.txt is "not done for search" and described it as a temporary crutch for coding tools (Ahrefs).

Do answer engines read llms.txt?

Very little. Ahrefs analyzed server logs and traffic for 137,000 domains using its analytics tool:

  • 28% of these domains publish an llms.txt file;
  • 97% of the files received no requests in May 2026;
  • for the files that were requested, 96% of requests came from bots;
  • 19.5% of requests came from identified AI tools, and 12% from tools studying llms.txt itself (GEO tools, checkers, researchers);
  • no AI bot requested an llms.txt file that did not exist: they do not look for it.

Among AI tools, GPTBot, OpenAI's training crawler, comes first, followed by Claude Code. Search bots that feed live answers, such as OAI-SearchBot and PerplexityBot, accounted for only 1.1% of requests. Ahrefs points out that a request proves the file was downloaded, not that it was used. The study covers Ahrefs' customers' sites, which are more technical than the web at large.

Should you create an llms.txt file?

It depends on who needs to read your site.

Yes, if you publish technical documentation, an API or a product used by developers. Coding agents are the actual readers of the file today. The AI labs themselves publish an llms.txt for their developer docs: OpenAI, Anthropic and Gemini, according to llmstxt.org.

Without expecting results, for a corporate site, a blog or an online store. The file will not make you appear in Google, and Ahrefs' data shows that answer engines' crawlers almost never request it. It is cheap to produce and harms nothing, but it does not replace what gets you cited: indexable pages, clear answers and sourced facts.

If you publish one, keep it current. An llms.txt describing old offers or old prices gives an agent false information about your business.


Sources

  1. 1./llms.txt, a proposal to provide information to help LLMs use websites, Jeremy Howard (Answer.AI) (3 septembre 2024)
  2. 2.The /llms.txt file, v2, llmstxt.org (mis à jour le 10 août 2026)
  3. 3.Google Search Central, guide d'optimisation pour les fonctionnalités d'IA générative
  4. 4.llms.txt, Chrome for Developers (audits de navigation agentique de Lighthouse)
  5. 5.We Analyzed 137K Sites: 97% of llms.txt Files Never Get Read, Ahrefs (15 juin 2026)
  6. 6.Google Confirms LLMs.txt Has No Current Implementation, Search Engine Journal (2 juin 2026)
  7. 7.Google Does Not Endorse LLMs.txt Files, Search Engine Roundtable (20 janvier 2026)

FAQ

Frequently asked questions

Does llms.txt improve Google rankings?

No. Google states that its Search does not use llms.txt files and that they have no effect on visibility or ranking, including in AI Overviews and AI Mode.

Do ChatGPT or Perplexity read the llms.txt file?

Rarely. In the Ahrefs study, live search bots such as OAI-SearchBot and PerplexityBot accounted for 1.1% of requests to these files, and 97% of files received no visits in May 2026.

What is the difference between llms.txt and robots.txt?

robots.txt tells crawlers what they are allowed to access. llms.txt describes the site's content and points to useful pages; it grants or denies no access.

Where should the llms.txt file go?

At the root of the site, at `/llms.txt`. Version 2 of the proposal also accepts a subdirectory, such as `/docs/llms.txt`, which then covers only the pages under that path.

Should I publish a Markdown version of every page?

The proposal recommends it for pages an agent may need: the Markdown version is published at the same address with a `.md` suffix and referenced with `rel="alternate" type="text/markdown"`. It is not required for the llms.txt file to be valid. ---

Read next

Raise your chances of being cited by ChatGPT and Google.

Free. Then a 3-day trial. Then Starter at €29 a month excl. VAT.

Start my 3 days, no card needed