Imagine a concrete order: an agency offers a small specialist company an llms.txt file. It is supposed to list the most important services, answers and links. As a result, the agency promises better visibility in AI Overviews and ChatGPT. The file is quickly explained; the claimed effect is not. “AI search” covers different products with different ways of finding web pages.
The sales promise behind llms.txt for SEO
The idea is reasonable at first: a short, machine-readable overview can show an agent which documents belong to a product and where the details are. The original llms.txtproposal describes exactly that — a Markdown file with background, orientation and links to further files. It is worded as a proposal. The file name does not oblige search engines to read it or to treat it preferentially.
The offer becomes problematic as soon as “can structure documentation” turns into “gets your page into AI answers”. Two things are missing for that: which service is supposed to use the file, and in which process? And how should the client recognise the claimed benefit? Without these answers, they may be paying for a nice-looking file instead of better findability of their actual content.
A clean content path can be useful. A placement in an answer cannot be derived from it.
What Google Search says about llms.txt
Google is unusually clear here. Its current guide to optimising for generative search features states that Google Search does not use llms.txt and similar special AI text files. According to Google, they neither help nor harm visibility in Google Search, including the generative features. A file may be discovered or indexed; that does not mean it serves as a special signal.
For AI Overviews, the search requirements apply instead: a page must be indexed and eligible to be shown as a supporting link. Even then, an appearance is not guaranteed. Crawler access, internal linking, readable main content and suitable answers to real questions remain the sensible checkpoints. If a service page is not indexed, investigate that cause first. An extra text file does not repair blocked crawl access or an unclear description of the service.
What remains open for ChatGPT, Bing and agents
Google’s statement applies to Google Search. It is not a verdict on every agent. The llms.txtinitiative describes one possible way of preparing documentation for agents. In software documentation with many technical subpages, a curated overview can serve a comprehensible purpose. Whether a particular system actually retrieves this file in the desired process has to be proven for that system.
For ChatGPT Search, OpenAI documents the search crawler OAI-SearchBot and control via robots.txt. This documentation does not mention any llms.txtrequirement for inclusion in search answers. That does not mean a ChatGPT agent could never read such a file; it only means that a guaranteed search effect is not proven. For its search and Copilot experiences, Bing likewise describes crawlability, indexing and clear content as the main foundations.
Anyone claiming that “all AI systems” automatically prefer llms.txt is lumping different products together without evidence. Anyone claiming the opposite, that the file is useless everywhere, goes beyond Google’s statement. Both are too rough for an investment decision.
The decision before you commission it
First ask for the purpose. Is it about help documentation that your own agent is supposed to load specifically? Then a small, maintained signpost can make sense. Is it about Google AI Overviews? Then, according to Google’s documentation, the promised effect is wrong. Is it about ChatGPT Search? Then the provider should explain why the file is supposed to be necessary there and distinguish between crawler access, mention and an actual website visit.
Also ask who is responsible for the content. A file, once published, can become outdated: prices, services and links change. Who maintains it, how often and based on which original pages? If these questions remain unanswered, the file is not a small SEO project but one more place for contradictory company information.
A verifiable brief could look like this: the provider names the target application, shows its documentation or a reproducible retrieval of the file and specifies which concrete user questions it should make easier to answer. The same questions are then used before and after the change to check whether the agent finds the right original pages. Even a positive test in one agent would not guarantee visibility in other products. It would only support the named use case.
Without such proof, a small inventory is usually more valuable: can the important pages be accessed without logging in? Do they clearly describe services, limits and how to get in touch? Are older documents and contradictory versions still offered as equivalent answers? These are questions a business can answer on its own website before ordering an additional file format.
A simple priority for small businesses: first, the important HTML pages must be accessible, understandable and up to date. After that, you can offer an additional summary for a clearly named agent. The difference is not technical but economic: you pay for a provable use case, not for the word “AI” in the offer.
