Large Language Model (LLM)

A large language model (LLM) is a neural network trained on very large amounts of text to predict the next token, and large enough that it can follow instructions, answer questions and write fluent prose without being trained separately for each task. ChatGPT, Claude and Gemini are assistants built on LLMs.

What the model knows and what it fetches

Training gives the model a compressed impression of the text it saw, stored in its parameters, fixed at a cutoff date and impossible for a site owner to inspect or correct. Retrieval gives it something else: at answer time the assistant searches, fetches a handful of pages, and writes from them. The two paths are run by different crawlers. OpenAI’s crawler documentation lists three product tokens with three jobs: one indexes pages for ChatGPT search, one collects training data, and one fetches a page on a user’s behalf. Blocking the training crawler in robots.txt changes what the model may learn from; it does not change whether ChatGPT search can fetch and cite the page.

Why an LLM names some businesses and not others

It writes from what it has. If the fetched page contains a passage that answers the question on its own and names the business consistently, that passage can be lifted and attributed. If the page contains a slogan and a form, the model fills the answer from other sources. This site’s own audit of 9,483 small-business sites found 61.5% with no question-and-answer content of any kind and 43.1% inconsistent about their own name, address or phone number across three pages; the breakdown is in what 9,483 small business websites serve to an AI crawler. Those are the pages the model reads.

A common question

Does an LLM remember my website from training?

Not in a form you can rely on or check. Training data carries no guarantee that a given site was included, no record of what was retained, and no way to update it before the next model. Retrieval is the path that can be measured, because the page the assistant fetches today is the page it writes from today.

Sources

Crawler roles: OpenAI’s crawler documentation. The 9,483-site figures are this site’s own, computed from its audit records; sample drawn per metro and trade, not from a random frame, and the percentages describe that sample.