Measuring AI Visibility: Methods, Metrics, and Common Mistakes

Why a single question proves nothing

If you want to know whether an AI recommends your company first, the instinct is to type in one question and see what comes out. That's understandable, but it's misleading. AI answers are inconsistent: ask the same question twice and you'll often get two different lists of names. Sometimes you show up, sometimes you don't. A single answer tells you less than you think — it's closer to a coin flip than a measurement.

There's a second problem: the question that occurs to you is rarely the one your customers actually ask. You think in terms of your product; customers think in terms of their problem. Someone looking for a hotel doesn't search for the hotel's name — they ask for something like "quiet, with a sauna, by the lake, dog-friendly." A real measurement needs many questions run many times, not one lucky hit.

The question panel: the heart of every measurement

The core building block is a fixed question panel — a set of a hundred to several hundred questions that covers your customers' entire decision journey, from the vague first search ("where can I find...") through comparisons ("which is better, A or B?") to direct questions about your name. Each stage measures something different.

The panel gets built carefully once, then stays untouched. That's not laziness — it's the basic condition for comparability. Only if you ask exactly the same questions every month can you put two months side by side. Change the panel and you're measuring something new from that point on; the time series breaks.

A good panel reflects how customers actually talk, not how marketing talks. It includes typos, casual phrasing, regional terms, and the small details that make a question real. The closer the panel is to real customer language, the more the score actually means.

Several AIs, several runs

There is no single AI. ChatGPT, Claude, Gemini, and Perplexity pull from different sources and answer differently. If you only measure one, you're only seeing a slice of your market. A solid measurement queries several assistants in parallel and keeps the results separate, so you can see exactly where you're strong and where you're invisible.

And because answers fluctuate, each question gets asked multiple times — typically three to five. Only the average across runs is a number worth building on. A measurement that asks each question once is mistaking chance for a result.

The three metrics that matter

Three metrics come out of these ratings, and together they give an honest picture. The first is the visibility score: a number from 0 to 100 that condenses how present you are overall. A mention counts, a recommendation counts more, and being named first counts most.

The second is the share of recommendations — often called share of voice. If a hundred questions surface three hundred company names total and thirty of them are yours, your share is ten percent. That number shows how much of the pie is yours and how much is going to everyone else.

The third is the breakdown by question type. A company can dominate questions about its own name while being completely absent from the category questions that actually bring in new customers. The overall score hides this; the breakdown exposes it and shows you exactly where the effort pays off most.

The most common pitfalls

The first mistake is the wandering question set. A better phrasing occurs to you, you swap it in, and you've just destroyed comparability. The rule: improvements go into a new, separate panel — the old one stays untouched for the time series.

The second mistake is treating a one-time snapshot as the truth. Without repetition, you're measuring noise, not a trend. The third is checking only your own brand and ignoring competitors — you see your own position but not the market around it. The fourth is confusing the score with revenue: it measures visibility, not bookings. It's a leading indicator, not a receipt.

• Changing the panel over time, then comparing months as if nothing changed
• Asking each question only once and taking the result at face value
• Measuring only one AI and missing the rest of the market
• Never measuring competitors alongside yourself
• Treating the score as a revenue forecast

Measure it yourself, or have it measured?

You can get a rough read yourself: ask your ten most important customer questions to two or three AI assistants, three times each, and note whether you show up. That's not a real measurement, but it tells you within half an hour whether you have a problem worth solving. Beyond that, method matters — a fixed panel, several AIs, several runs, documented answers, and a time series tracked over months.

The real value of measuring isn't the number itself — it's the to-do list it produces. Every question where you're missing tells you exactly which answer doesn't exist on the web yet. That turns a diagnosis into a work plan, and turns the score from a vanity metric into an actual tool.

What counts as a good visibility score?

There's no single number that counts as "good," because it depends entirely on the market. In a crowded field with many well-known providers, a score of 30 can be strong. In a niche with little competition, that same 30 might be weak. The score only means something in comparison — against your own past months, and against competitors measured in the same panel.

The trend matters more than the absolute number. A business that climbs from 9 to 40 over six months has achieved more than one that sits flat at 45. The first is building momentum; the second is just holding a position. Anyone reading the score should ask two questions: where do I stand relative to whoever's winning these same questions, and which direction is my curve pointing?

The distribution behind the score matters just as much. Two companies can both post a 35 while sitting in completely different positions — one is faintly visible everywhere, the other is front and center on a handful of questions and invisible on the rest. That difference determines what to do next, and only the breakdown reveals it.

Manual spot-check or systematic measurement

There's a real gap in effort — and in payoff — between a quick self-check and a full measurement. The manual spot-check takes half an hour and answers one question: do I have a problem at all? For that, it's enough. Beyond that it's too coarse, because it doesn't smooth out the fluctuation or capture how things change over time.

A systematic measurement does that work for you and makes it comparable. It runs the same panel every month, logs every answer verbatim, and builds a time series you can actually read cause and effect from. The real payoff isn't the automation — it's the discipline. Because the conditions stay constant, you're measuring real progress instead of the day's noise.

For most companies the sensible order is: run the manual spot-check first to decide whether the problem is worth solving, then move to systematic measurement as the basis for the actual work.

Why competitors belong in every measurement

Measuring your own visibility alone is like timing your own run without knowing anyone else's time. A number with nothing to compare it to says very little. Only when competitors are measured in the same panel does a real picture of the market emerge — who gets recommended on which questions, who shows up everywhere, and where the field is surprisingly empty.

That view changes your priorities. Questions where a strong competitor already dominates the answer are hard to win and can wait. Questions where no one is really visible are open doors — a good answer page is often all it takes to be first to claim the spot. Without competitor data, you miss these openings and spread your effort evenly instead of putting it where it counts.

In practice: include three to six relevant competitors in the panel from the start and measure them against the same questions. The extra comparison costs almost nothing, but it turns the score from an isolated number into a map that shows you exactly where the next move pays off most.