Benchmark

A benchmark is a reference value that a measurement is judged against: a figure from a competitor, a sample average, or a fixed set of tests that every system is run on. It comes from outside the thing being measured, which is what separates it from a baseline.

Two meanings that get mixed

In marketing, a benchmark is a peer figure: how the other businesses answering the same customer questions are doing. In AI research it is a fixed evaluation set. Aggarwal et al., “GEO: Generative Engine Optimization” (KDD 2024), built one for generative engines, a set of queries against which source-side changes could be scored, so that a gain in visibility could be attributed to the change and not to a different question. The two meanings share a property. A benchmark only works if it stays still while the thing measured against it moves.

What a fair AI visibility benchmark requires

The same questions, put to the same assistant, on the same day, for every business in the set. Vary any of those and the comparison is between conditions, not businesses. The closest this site has to a sample-level benchmark is its one-question probe of 9,240 small businesses in September 2026, run against Claude with retrieval: 17.7% were named among the 5,928 answers that gave a differentiated list rather than a hedge. It is directional, one assistant, one question, one day. The details are in what 9,483 small business websites serve to an AI crawler.

For a single business the working benchmark is closer to home. Every panel answer that does not name the business names someone else, and the list of who was named instead, on which questions, is the peer figure that matters. It is recorded as part of the score.

A common question

What counts as a fair peer comparison for one business's AI visibility?

The same questions, put to the same assistant, on the same day, for every business in the set. Vary any of those and the comparison is between conditions rather than businesses. For a single business the working peer figure is closer to hand: every panel answer that does not name the business names somebody else, and the record of who was named instead, on which questions, is kept as part of the score. A benchmark only does its job while it holds still and the thing measured against it moves.

Sources

The evaluation-set sense: Aggarwal et al., “GEO: Generative Engine Optimization” (KDD 2024). The probe figures are this site’s own; sample drawn per metro and trade, not from a random frame.