Do I absolutely need an llms.txt?

No, it is voluntary and not yet a binding standard. But it is a cheap, low-risk building block: whoever takes AI visibility seriously gives language models a clean orientation to their own core content with it.

llms.txt

Q: Is llms.txt the same as robots.txt?

No. robots.txt controls what crawlers are allowed to read at all. The llms.txt actively suggests to language models which content is especially important. The two complement each other but don't replace one another.

llms.txt is a proposed plain-text file, placed at a site's root (/llms.txt) and written in Markdown, that lists a site's key pages with one-line summaries so a language model could, in theory, skim it instead of crawling the full site. It is a voluntary convention, not a standard any major AI system has confirmed it reads. No search engine or AI assistant is required to fetch it, and none has stated that it factors into rankings or citations.

Why llms.txt gets attention

The pitch is intuitive: HTML pages carry navigation, ads, and scripts that make them noisy for a model to parse, so a clean Markdown digest sounds like it should help. That intuition is why llms.txt spread quickly after it was proposed. But intuition isn't evidence. Google's John Mueller has confirmed that no Google Search system reads or acts on llms.txt files, and Google's own guidance says the file neither helps nor hurts rankings. An Ahrefs analysis of roughly 137,000 sites that published an llms.txt found about 97% saw zero measurable referral traffic tied to it. Treat the file as a hedge with a small setup cost, not a lever with a demonstrated payoff.

How llms.txt works

The file lives at yourdomain.com/llms.txt and follows a loose Markdown convention: a heading naming the site, a short description, then grouped link lists where each entry pairs a URL with a one-sentence summary. Some sites also publish an llms-full.txt containing the full body text of key pages, so a crawler wouldn't need to fetch each URL separately. It doesn't replace your XML sitemap or robots.txt file — those still govern crawl access and page discovery through established, machine-enforced protocols. llms.txt is a proposal aimed at language models specifically, and as of today no major AI provider — not OpenAI, not Anthropic, not Perplexity — has confirmed using it as an input to citations or answers.

Common mistakes

The most common mistake is treating llms.txt as if it were confirmed to work: teams spend hours curating it expecting a citation or ranking lift that the evidence doesn't support. A second mistake is dumping the entire sitemap into it, which defeats the file's only plausible value — a curated, human-written summary rather than a raw URL list. A third is linking pages that have since moved or been deleted, since nothing enforces freshness the way a sitemap's protocol does. If you publish one, keep it small, accurate, and current, and don't let it substitute for the things that are actually evidenced to matter: clear on-page answers, structured data where it fits, and being mentioned accurately elsewhere on the web.

Relation to AI citation

Whether a model cites or names your brand correctly has little to do with whether you have an llms.txt file. Ahrefs' analysis of roughly 75,000 brands found that how often a brand is mentioned across the web correlates with AI citation rate at about 0.664 — roughly three times stronger than the correlation with backlinks, at about 0.218. Separately, Ahrefs found that only 6 to 8% of URLs cited by ChatGPT overlap with Google's top-10 results for the same query, and about 80% of ChatGPT-cited URLs don't rank in Google's top 100 at all — AI citation runs on a different selection process than search ranking, and llms.txt isn't a confirmed part of either. If you want to influence citation odds, the evidenced levers are third-party mentions and content that directly answers a question, not a root-level manifest.

Example

Picture a small tax firm in Leipzig that reads about llms.txt and adds one at its-domain.com/llms.txt: a one-line description of the firm, then grouped links to its services page, a few guides, and its about page, each with a short summary. It's a reasonable half-day task and it costs nothing to have. What it won't do, based on the current evidence, is make ChatGPT or Google's AI Overviews suddenly cite the firm — that depends far more on whether other sites mention the firm by name and whether its pages answer client questions plainly.

Common questions

Does llms.txt improve AI Overviews or ChatGPT citations?

There's no confirmed effect either way. Google has stated no special file is needed for AI Overviews or AI Mode and explicitly warns against writing content just "for AI." No major AI provider has confirmed reading llms.txt as a citation input, and an Ahrefs study found about 97% of sites publishing one saw no measurable referral traffic from it.

Is llms.txt the same as robots.txt?

No. robots.txt is an established, machine-enforced protocol that controls what crawlers may access. llms.txt is a voluntary proposal suggesting which content matters most to a language model, with no confirmed mechanism reading or acting on it. The two aren't interchangeable and llms.txt doesn't carry robots.txt's enforcement.