Setting up llms.txt: what the file is, what it is not, and where it ranks

What is llms.txt?

A plain markdown file at the root of a site, /llms.txt, that tells a language model what the site is and which pages to read. The llmstxt.org proposal fixes the shape: an H1 with the site or project name, a blockquote holding a short summary, then optional sections, each an H2 followed by a list of links with a few words on what each page contains. The idea is that a model handed a whole site's HTML spends most of its context window on navigation and scripts, and a curated list of the pages that matter, in markdown, is easier to take in.

It was written with documentation sites in mind, where a model helping a developer needs the reference pages and not the marketing. For a small business the same logic applies at a smaller scale: the service pages, the price page, the contact page and the about page are what an assistant answering a buyer needs; the careers page and the cookie notice are not. The glossary entry on llms.txt holds the definition. This page is about whether to publish one, and how.

What llms.txt is not

It is not a standard, and nothing that reads it has said so. Three facts, stated plainly. The proposal calls itself a proposal. None of OpenAI, Anthropic or Perplexity has published documentation saying its crawler or its assistant fetches /llms.txt or gives weight to what it says; what they document is product tokens and robots.txt (OpenAI, Perplexity), as Google does for its own crawlers (Google). Google also says it outright: its guide to generative AI search says Google Search ignores llms.txt, so keeping one for other services neither helps nor harms a site there. And Google's page on AI features and your website says no special file or markup is required for AI Overviews or AI Mode: a page has to be indexed and eligible for a snippet, which is the ordinary condition.

It is also not robots.txt, and it does not replace it. robots.txt is read under RFC 9309 by every crawler whose operator publishes documentation; it grants or refuses access, and it is where the answer-side tokens are allowed in or kept out. llms.txt grants nothing and refuses nothing. A site that blocks OAI-SearchBot in robots.txt and publishes a careful llms.txt has told ChatGPT search to stay out and then left it a reading list. robots.txt for AI crawlers covers the file that does bind.

And it is not a ranking signal, a citation guarantee or a way to correct what an assistant says. We have no measurement that a site with the file is named more often than a site without one, and we would not expect to find one while nothing documents reading it.

What to put in it, if you publish one

The truth about the business, short, with links to the pages that carry the detail. Write it for a reader who has one minute and no browser: what you do, where, for whom, what it costs, how to reach you, and then the list. Every page you link should be one you would want quoted, which means the answer-shaped pages, not the blog archive. Here is the shape for a small business. Replace the placeholders and delete any section you have no pages for; the same file, line by line with what each part is for, is on our fill-in llms.txt template.

# [Business name]

> [Trade] in [Town] and [surrounding area]. [One sentence on what
> you are known for.] Open [days and hours]. [Phone]. [Street address].

Last updated: [YYYY-MM-DD]

## Services

- [Service page A](https://www.example.com/services/a): what it is,
  who it is for, the price range
- [Service page B](https://www.example.com/services/b): the same
- [Emergency and after-hours](https://www.example.com/emergency): when
  you answer, what the call-out costs

## Prices and policies

- [Pricing](https://www.example.com/pricing): what is a flat fee, what
  is quoted on site, how estimates work
- [Guarantees and warranties](https://www.example.com/guarantee)

## About

- [About us](https://www.example.com/about): founding year, the people,
  licenses and memberships by number
- [Reviews](https://www.example.com/reviews): where the ratings come
  from, and how many there are
- [Contact](https://www.example.com/contact): one address, one phone
  number, hours by day

## Optional

- [Articles](https://www.example.com/articles): what you write about

Two rules while you fill it in. The facts in the blockquote must match the site and the markup exactly: the same phone number, the same address, the same hours, because a file that disagrees with the contact page is one more contradiction for a machine to resolve. And date it, in the file, so a reader can tell whether it is current; ours carries a last-updated line under the summary.

Keep it short. The file is a table of contents, not a copy of the site. If you find yourself pasting the service page into it, link the service page instead and make sure the service page is quotable.

Where it goes, and how to check it is served as text

At the root of the host the site serves from, as plain text: https://www.example.com/llms.txt, returned with a 200 status and a Content-Type of text/plain or text/markdown. Not a page that renders markdown. The file itself.

1. Save the file as llms.txt, UTF-8, with no HTML in it, and upload it to the web root next to robots.txt. On a managed platform, look for the setting that serves static files at the root, or ask your host. Some platforms route every unknown path to the homepage, and those serve an HTML page under the name, which is worse than no file.
2. Fetch it the way a machine would: curl -i https://www.example.com/llms.txt. The first line should say 200. The Content-Type header should say text/plain or text/markdown. If it says text/html, your platform is serving a page, not the file.
3. Read the body of the response, not the browser view. It should begin with your # heading. If it begins with <!DOCTYPE, go back to the first step.
4. Check it on the host your site resolves to. A file at example.com/llms.txt does nothing for www.example.com, and the other way round.
5. Confirm robots.txt does not block it. It is a path like any other, so a Disallow: / written for an answer-side token covers it too. Which is a reminder of which file binds.

What the crawl found

Almost nobody has one, and that is the least important finding in the audit. In our audit of 9,483 small business websites (40 trades, 90 metros, September 2026), 63.6% published no /llms.txt. We listed that figure last in the write-up on purpose. The findings above it describe things that stop an AI crawler reading or quoting a site: 61.5% with no question-and-answer content, 64.7% with no review signal a crawler could read, 43.1% inconsistent about who and where they are, 17.8% with no schema markup at all, 7.2% behind bot protection that served our crawler a challenge page. A site with every one of those problems and a perfect llms.txt has a reading list pointing at pages nothing can use.

What the figure does tell you is whether anyone at the business has thought about a machine reader at all. The figure counts whether the file was there, not what it said, and a file that lists pages which no longer exist, or states an old phone number, is a signal that cuts the other way.

Where it ranks against the rest of the work

Last, after the things that are documented to matter, and it is an hour of work once those are done. The order below is the order in which a missing piece stops an assistant.

If you have done the first four, write the file. It takes an hour, it is one more place the truth about the business is stated in a form a machine can take in, and if an operator ever documents reading it, you are already there. If you have not done the first four, the hour is better spent on the second or the third.