Does anyone read llms.txt?

No crawler I can verify has read this site’s llms.txt. In 63 days of IP-verified logs, seven AI and search agents made 2,456 verified requests, and none of them was for /llms.txt or /llms-full.txt. In the same window, those agents read the markdown twins of my posts 75 times.

The file has been live since 27 July. I said in logging AI crawlers that it had zero fetches and that the count would get its own post when ten weeks made zero a fair number. This is that post.

How I counted requests for llms.txt

The site’s middleware logs every request whose user agent matches one of 30 named AI and search agents. The matcher has covered /llms.txt since the logger went live on 29 July, so a request for the file from any of those agents gets a row. Since 10 August each row also carries the client IP and a verdict: verified when the IP is inside the operator’s published ranges, forged when the operator publishes ranges and the IP is outside them, unknown when the operator publishes none.

I exported the log on 12 October: 6,063 records from 29 July to 12 October, 75 days. Two of them are requests for /llms.txt. Both are mine: they are the smoke tests I sent when the logger went live, each one part of a set of three requests sent in the same second, 45 minutes apart. Without them, the count for the full 75 days is zero.

The rest of this post uses only the 63 days from 10 August, because the 902 records before that date carry no IP and can never be verified. I report each agent on its own, not as a total share, because only some operators publish IP ranges.

Zero verified fetches of llms.txt in 63 days

ClaudeBot asked for robots.txt 615 times and for the sitemap 611 times. A crawler that checks two files at the site root that often knows how to ask for a file at the root. It did not ask for this one.

Unverifiable agents give the same answer. Among the 5,161 rows since 10 August, including every unknown and forged one, not one is a request for /llms.txt or /llms-full.txt. Meta’s crawler made 1,436 requests for pages and other paths in that window, and Amazonbot 328. None of them asked for either file.

The markdown twins are found through the HTML page, not the index

Every post here has a markdown twin at its own path with .md added. Each HTML page links to its twin with <link rel="alternate" type="text/markdown">, and llms.txt also links every twin. In 63 days the twins got 144 requests across 31 pages. 75 were verified: ClaudeBot 31, GPTBot 29, OAI-SearchBot 9 and Bingbot 6. Meta’s crawler made 39 more and Amazonbot 30, which I cannot verify.

The index cannot be how the crawlers found the twins, because nothing read the index. The log shows the other route. All 75 verified twin requests came after the same agent had read the HTML page for the same post, and 69 of them came within 24 hours of that read. The crawler reads the page, sees the alternate link in its head, and asks for the markdown.

That route is also fast. Of the 19 posts published since 11 August, 17 had their twin read by a verified ClaudeBot or GPTBot on the day they were published. The other two were read the next day.

What Ahrefs found across 137,210 sites

Ahrefs looked at the same question on 137,210 domains in its web analytics product during May 2026. 28% of them published an llms.txt file, and 97% of those files got no requests at all in the month. The roughly 1,100 files that did get requests shared about 22,000 of them, and 96% of those came from bots. Named AI tools made 19.5% of the requests, with GPTBot first and Claude Code second.

My site is one of the 97%. The difference is in who would have asked. In Ahrefs’ data GPTBot is the AI agent that fetches llms.txt most. On this site GPTBot made 210 verified requests in 63 days, read 29 markdown twins, and never asked for the index. Ahrefs also gives a warning that applies to my count: a fetch is not a read. My zero does not need that warning; a request that never happened was not read.

What this count cannot see

The logger matches 30 user agents and nothing else. Three groups of clients that could fetch llms.txt are outside it:

• Coding agents. Claude Code is the second-largest AI fetcher in Ahrefs’ data, and its user agent is not on my list. If a developer pointed an agent at this site, my log would not show it.
• SEO audit and profiling tools. SEO audit tools are the largest group of fetchers in Ahrefs’ data, with 21.7% of requests, and tech profiling tools made 11.6%. Neither group is on my list.
• People. A human who opens /llms.txt in a browser is not logged.

So the finding is narrow and exact: the 30 AI and search crawlers I log never asked for the file. It is not a finding that nobody ever opened it.

Why I keep llms.txt anyway

The file costs almost nothing. It is 8,218 bytes with 51 links, generated at build time from the same content collection as the sitemap, and checked by the same gate that stops drafts from leaking into it. llms-full.txt is 235,132 bytes, also generated, and noindexed. Deleting them would save no work, and it would end the measurement. The year-end essay will count again.

What I would change in a new build is the order of the work. The markdown twins and the rel="alternate" link in each page head got 75 verified reads in nine weeks. The index file got zero in the 75 days I have logged. If you want AI crawlers to read a clean version of your pages, link it from the page they already fetch. The file at the root is the part nobody has asked for.