The argument you see most often for publishing llms.txt is that Anthropic and OpenAI publish one.
That reasoning runs the wrong way round. Anthropic hosting llms.txt on its developer documentation tells you the file is useful for coding agents loading API docs. It tells you nothing about whether ClaudeBot reads the llms.txt on your company website during retrieval. Those are different systems doing different jobs, and conflating them is most of why this topic stays confusing.
Here is what the file is, what the evidence actually shows, and when it is worth twenty minutes.
What it is
A markdown file at /llms.txt containing a title, a short summary, and curated links to the pages on your site worth reading. Jeremy Howard of Answer.AI proposed it in September 2024, and the spec reached version 2 in August 2026.
The idea is reasonable: give an agent already reading your site a fast route to the right page, without it wading through navigation, scripts and cookie banners.
llms-full.txt is the heavier variant, containing the actual content rather than links to it.
What the major providers have said
| Provider | Position |
|---|---|
| Gary Illyes at Search Central Live: does not support it, no plans to. John Mueller compared it to the keywords meta tag | |
| OpenAI | Documented crawler control is robots.txt. No public commitment to read llms.txt as a retrieval signal |
| Anthropic | Publishes llms.txt for its own docs. Has not confirmed ClaudeBot consumes it as a retrieval signal |
| Perplexity | Publishes site-owner guidance covering robots.txt and user agents, not llms.txt |
John Mueller's Bluesky post in June 2025 was blunt: no AI system currently uses llms.txt, and it is obvious from server logs, because the consumer chatbots fetch your pages but not the file.
As of 2026, no major provider, including Google, OpenAI, Anthropic, Meta and Mistral, has publicly committed to reading or acting on llms.txt in production.
The adoption number, read properly
An SE Ranking study of 300,000 domains found a 10.13 percent adoption rate.
The interesting part is the distribution: roughly 9.88 percent on low-traffic sites, 10.54 percent on mid-traffic, and 8.27 percent on the largest. Essentially flat, with the biggest sites slightly less likely to have one.
If llms.txt conferred visibility advantages, you would expect sophisticated high-traffic sites to adopt it fastest. They have not.
There is also a reason adoption is not zero that has nothing to do with belief in the standard. When Mintlify shipped llms.txt support across every documentation site it hosts in late 2024, thousands of sites gained the file overnight, Anthropic and Cursor among them. A good share of adoption is a platform default, not a decision.
The one genuine conflict in the evidence
Not everything points the same way, and the disagreement is worth stating rather than resolving.
Profound, which sells GEO tracking, reported that crawlers from Microsoft and OpenAI are actively fetching llms.txt and llms-full.txt. Limy, analysing over 500 million LLM bot traffic events, reported that AI search crawlers almost never fetch /llms.txt.
Both are vendors in this category. Server logs are the only way to settle it for your own site, and checking yours takes a minute: grep your access logs for llms.txt and see whether anything has requested it.
Separately, Google included an llms.txt file in its Agent-to-Agent protocol, and the Chrome team shipped an llms.txt check in Lighthouse's experimental audits. Both concern agentic browsing rather than search ranking, which is a distinction the coverage usually loses.
When it is actually worth doing
The useful framing is that llms.txt is a business-to-agent convention, not a search optimisation one.
Worth doing if you publish developer documentation. This is the real use case and the only one with evidence behind it. A developer working in Cursor or [Claude Code](/tools/claude-code "autolink") pointing an agent at your docs genuinely benefits from a curated markdown map. If you run a docs site, publish one.
Worth doing if agents interact with your product. As agentic browsing grows, a file that helps an agent navigate your site has plausible future value. Cheap insurance.
Not worth doing for rankings or citations. There is no public evidence that publishing llms.txt raises visibility in ChatGPT, Perplexity, Claude or Google answers. Anyone selling it as a GEO lever is ahead of the evidence.
The honest cost-benefit: it takes twenty minutes, it cannot hurt, and it will probably do nothing for your AI visibility. Do it if that framing appeals. Do not do it instead of work that has evidence behind it.
One risk worth knowing
Because llms.txt lets you present a different version of your content to agents than to users, it creates a cloaking surface. Mueller raised this directly.
Nobody is enforcing against it today because nobody is reading the file. If adoption ever arrives, sites that used it to show bots something other than their real content will be the first problem the standard has to solve.
Keep yours an honest map of pages that genuinely exist and say what you claim.
What to do instead
If the goal is being cited in AI answers, the evidence points at unglamorous things: being crawlable, publishing information that exists nowhere else, being a recognised brand, and writing clearly enough to be quoted.
llms.txt is a twenty-minute task that feels like progress. The things that actually move AI visibility take months and are harder to point at in a status update, which is probably why this file gets more attention than its evidence supports.
FAQ
Is llms.txt a Google ranking factor? No. Google has stated publicly that it does not support llms.txt and has no plans to.
Does llms.txt improve AI citations? There is no public evidence that it does. No major answer engine documents it as a citation signal or visibility requirement.
Should I create one anyway? If you publish developer documentation, yes. Otherwise it is optional, cheap, and unlikely to do anything measurable.
What is the difference between llms.txt and robots.txt? robots.txt controls crawler access and is respected by every major AI crawler including GPTBot, ClaudeBot, Google-Extended and PerplexityBot. llms.txt is a content map with no access-control function and no confirmed consumer.
Do I need llms-full.txt as well? Only for documentation sites where an agent benefits from loading the whole corpus. For most sites it is unnecessary weight.
How do I check if anything is reading mine? Search your server access logs for llms.txt. That answers the question for your site better than any published study.
How this guide was researched
Desk research, cross-checked across published studies, vendor analyses and direct statements from Google representatives in 2026.
Where the evidence conflicts, specifically on whether AI crawlers fetch the file, both positions are stated along with who published them. Both parties sell products in this category.
Positions can change. If a major provider ships a retrieval pipeline that reads llms.txt, the recommendation here changes with it.
