If you run a service business — a law firm, a clinic, a car service, an agency — you do not need an llms.txt file. It will not get you cited in ChatGPT, it will not help you in AI Overviews, and Google has said in writing that Search ignores it.
If you sell software with technical documentation, the answer changes. The file does have a real audience. It is just not the audience the SEO industry promised.
Here is the data, the distinction, and what to do with the time you were about to spend on this.
What llms.txt Was Supposed to Do
Jeremy Howard of Answer.AI proposed the format in September 2024. The problem he described was real: AI systems burn effort crawling full HTML pages stuffed with navigation, scripts, and ads to find a few paragraphs of actual content. A short markdown file at yoursite.com/llms.txt could point them straight to what matters, in a format they parse cleanly.
Nothing in that proposal promised search visibility. Ahrefs put it directly in their analysis: the «AI visibility» framing came later, attached by the SEO industry on the speculation that AI platforms would reward the file.
The confusion is understandable. The name looks like robots.txt and sitemap.xml — two files that genuinely shape how search engines treat your site. It is an easy leap to assume llms.txt works the same way for AI search. No AI vendor has ever documented that it does.
Adoption jumped anyway, mostly for reasons unrelated to results. Yoast added automatic llms.txt generation. Wix rolled the file out globally. When plugins with millions of installs turn a feature on by default, adoption numbers climb whether or not the feature does anything.
What the Data Shows
Three independent datasets, same conclusion.
Ahrefs analyzed 137,210 domains in May 2026. Of the sites that had published an llms.txt file, 97% received zero requests for the entire month. Not one fetch, from any bot or human. Of the 3% that did get traffic, most requests came from bots that were not AI tools at all. The full breakdown sits behind Ahrefs’ llms.txt analysis.
The detail that says the most: in Ahrefs’ data, Slackbot fetched llms.txt files more often than PerplexityBot. A chat app’s link-preview crawler beat one of the AI search engines the file supposedly exists to serve.
Otterly.AI ran a 90-day study. Across more than 62,000 AI bot visits, 84 touched an llms.txt file. That is 0.1%.
ALLMO checked citations rather than crawls. Out of 94,614 URLs cited in AI answers, exactly one was an llms.txt file. SE Ranking found no citation correlation at all across roughly 300,000 domains.
Crawl data, citation data, and correlation studies all land in the same place. The file is published widely and read almost never.
What Google Says, and What Google Does
On May 15, 2026, Google published its first official guide to optimizing for generative AI features in Search. It includes a mythbusting section listing things site owners do not need to do. llms.txt is on that list.
Google updated the guide again in June with language that leaves no room for interpretation: you do not need to create machine-readable files, AI text files, markup, or Markdown to appear in Google Search or its generative features, because Search does not use them. Maintaining one will neither harm nor help your rankings.
John Mueller has compared the file to the old meta keywords tag — a site’s unverified claim about itself, which search engines learned to discount decades ago.
Then there is the other half of Google. Days before that guide went live, Chrome shipped Lighthouse 13.3 with a new Agentic Browsing audit category, and one of the checks looks for an llms.txt file and flags errors retrieving it.
Same company, same month, opposite signals. It is not sloppiness — it is two teams solving different problems. Search does not use the file for ranking or citations. Chrome’s team is thinking about AI agents navigating sites on a user’s behalf, where a machine-readable index is genuinely useful. That split tells you exactly what llms.txt is: infrastructure for agents, not a visibility lever for search.
Who Actually Reads It
The file has a real user base. It is coding tools.
Claude Code, Cursor, GitHub Copilot, Windsurf and similar assistants look for /llms.txt when a developer points them at a product’s documentation. It saves them from parsing a docs site’s HTML shell to find API references. Ahrefs’ own data reflects this — among the small share of AI bots that fetched the file, the standouts were GPTBot, which collects training data rather than answering live queries, and Claude Code, a coding agent.
The bots that answer questions in AI search products — PerplexityBot, OAI-SearchBot — accounted for around 1% of requests. And the user-triggered bots, the ones that fetch a page because someone asked ChatGPT or Perplexity a question right now, fetch the specific page in question. They do not stop at a directory file first.
So llms.txt works. It works as documentation delivery for developer tooling. That is a legitimate use case, and if it describes your product, publish one. It is simply not the use case anyone was selling.
Does Your Business Need One? Three Questions
Most of the writing on llms.txt comes from software companies talking to other software companies, which is why the advice never quite fits a local or service business. Here is the version that does.
- Do you have technical documentation that an AI coding agent would fetch?
API docs, SDK references, developer guides, integration instructions. If yes, llms.txt has a real job on your site: helping those tools find and read your docs cleanly. If your site is service pages, a blog, and a contact form, there is nothing for an agent to index this way.
- Is it already being generated automatically?
If Yoast, Wix, or a plugin creates the file for you, leave it. It costs nothing, breaks nothing, and Google has confirmed it does no harm. Just do not add it to a reporting dashboard or treat it as work completed toward AI visibility. Nothing was accomplished.
- Have you already done the work that actually drives AI citations?
This is the real question. llms.txt takes ten minutes, which is exactly why it gets done first — it feels like progress on a problem that otherwise requires months of content work. If your service pages are thin, your content has no specific numbers in it, and your structured data is broken, a markdown index of those pages changes nothing. You would be handing AI systems a cleaner map to content they were never going to cite.
For the overwhelming majority of service businesses, the honest answer across all three is no, no, and not yet. Skip the file.
What Actually Gets You Cited in AI Search
Google’s own guidance points back to the fundamentals, and the citation data agrees. Four things move the needle:
Specific, extractable answers. AI systems quote pages that commit to concrete information — prices, timeframes, distances, steps. Pages that talk around a topic without answering it get skipped. This is the core of Answer Engine Optimization, and it is where most sites lose before any technical question comes up.
Clean structure. Clear headings, short paragraphs, direct answers under the question they answer. AI systems extract from structured content far more reliably than from long narrative blocks.
Structured data on real content. Schema markup helps AI systems parse what your page contains. Google says it is not required for generative AI features and warns against over-indexing on it, but it remains useful — as we covered in our breakdown of what FAQ schema still does after Google retired FAQ rich results.
External authority signals. Reviews, directory profiles, brand mentions, links from relevant sites. AI systems weigh what the rest of the web says about you. A technically perfect site with no external footprint stays uncited.
None of this takes ten minutes. That is precisely why llms.txt got popular.
Want to Know What Is Actually Blocking Your AI Visibility?
We run a free AEO audit. We look at how extractable your key pages are, check your structured data, and identify what is genuinely holding you back from being cited in AI search — not a generic checklist, specific pages and specific fixes. If you want help implementing them, that is what our SEO services cover.
No commitment. You will know where you stand in a few days.
TwoX Digital is a digital marketing agency specializing in SEO, AEO, GEO, and PPC for businesses in the US and English-speaking markets.
