Does llms.txt work?

An llms.txt file is a Markdown list of your most useful pages, placed at the root of your site. The idea is that an AI system reads it instead of crawling everything. The idea makes sense. The measurements say the major AI search systems mostly don’t ask for it.

Does Google use llms.txt?

No. Google’s guide to generative AI features says Search doesn’t use llms.txt or other AI text files, and that you don’t need to create one to appear in its generative AI features. To be shown in AI Overviews or AI Mode, a page must be indexed and eligible to show in Search with a snippet, and the site must be included in Search generative AI features in Search Console.

That covers Google only. It says nothing about ChatGPT, Claude or Perplexity, so the next sections use their traffic.

Do AI crawlers read llms.txt?

Rarely. Three studies looked at crawler logs and domain data. They agree.

Study	What it measured	Result
Ahrefs, May 2026	137,210 domains. About 38,000 had a valid llms.txt.	97% of those files got zero requests in the month. 28% of the domains published one.
ByDefault, 90 days to 12 Oct 2026	1,877,319 AI bot requests across its sites	4,647 asked for llms.txt. GPTBot, ClaudeBot, OAI-SearchBot and PerplexityBot asked 85 times, against 12,059 requests for robots.txt.
SE Ranking	About 300,000 domains, 10.13% with the file	No relationship between having the file and being cited. A model of citations became more accurate when llms.txt was removed as an input.

Ahrefs says its customers skew technical, so 28% is probably an upper bound for the web as a whole. ByDefault’s numbers come from its own crawler logs and tracked prompts, so they reflect its sites and its prompt set. Neither study can tell you what a crawler did with a file after fetching it.

OpenAI’s own crawler documentation covers robots.txt controls for OAI-SearchBot and GPTBot. It publishes an llms.txt for its developer docs, but nothing on the page says its crawlers read one.

Do AI answers cite llms.txt files?

Almost never. ByDefault ran tracked prompts against eight assistants between 15 July and 12 October 2026 and recorded 108,417 answers. 303 cited an llms.txt file, which is 0.28%.

Assistant	Answers	Citing an llms.txt
Claude	17,719	24
Google AI Overviews	6,437	5
ChatGPT	17,733	9
Gemini	5,329	0

The table shows four of the eight assistants. Those four account for 38 of the 303. The coding assistants in the same sample cited it most, at 0.43% of answers across the four coding assistants. A developer asking a coding tool about a library is a different job from a buyer asking for a shortlist.

Who does read llms.txt?

Agents that fetch pages for a specific person. In ByDefault’s logs ChatGPT-User asked for llms.txt 3,141 times, and 3,130 of those were on a single developer documentation site. Claude-User made 761 reads, 655 of them on that same site. Without it, the numbers are small.

So the file is useful in one place: documentation that a developer’s AI tool opens while building something. If you sell software with an API or a public docs site, an llms.txt there has a real audience. On a marketing site that answers “which tool should I buy”, the studies show no audience.

What did we see on manufacturer sites?

We checked 66 manufacturer websites on our Ontario target list on 13 September 2026, and 65 were reachable. 25 publish an llms.txt. Seven of those were generated automatically by a WordPress SEO plugin, Yoast or All in One SEO, so the owner may not know it exists. One is a hand-written file that addresses shopping assistants directly.

We also checked the 37 scored companies in our AI Visibility Index. They were scored out of 100 between 28 July and 13 September. Ten of them publish llms.txt.

Group	Companies	Mean score	Median	Range
Publishes llms.txt	10	56.1	59.5	32 to 67
Doesn’t	27	51.6	54	27 to 74

The gap is 4.5 points on a small sample. Two things weaken it as evidence. The top score in the whole set, 74, belongs to a company without the file. And our site-readability dimension can credit a clean llms.txt, so some of that gap may be our scoring at work. Read it as “no sign it matters”.

One more case in the Index: a manufacturer whose robots.txt blocks three of the four main AI engines, while it publishes an llms.txt. It is still named in AI answers. The file didn’t cause that. It doesn’t fix the blocking either. The file and the access rules are separate things.

Should I add one anyway?

Google says you can ignore it for Search. In our view it does no harm. Add it if you publish developer docs, or if you want a clean file for tools that read it. Keep it accurate, since a file that lists dead pages is worse than none.

Don’t treat it as a plan. These are the things that decide whether an engine names you:

• OAI-SearchBot and the other search crawlers can fetch your pages. OpenAI says sites opted out of OAI-SearchBot won’t appear in ChatGPT search answers.
• Your key facts are in the HTML, in text, not locked in PDFs or images.
• Independent pages, such as directories, review sites and trade coverage, say the same thing about you.

Geongyn, Bayham (Vienna), Ontario, Canada, checks those three things for B2B SaaS companies and building-product manufacturers. Book a teardown.