Measuring AI visibility for clients: how an agency grades its own work

How can an AI visibility report flatter a month without anyone lying?

At five points, each of them ordinary. The agency picks the questions, keeps or changes them, runs them, reads the answers, and explains what moved, and it bills for movement. Nobody has to lie for a month to come out better than it was. A question that never names the client gets reworded, a passing mention gets marked as a recommendation, and a rise gets credited to the agency's page when the client's own team changed the site that week too.

The rules are not an accusation. They make the report something the client can check alone, which is also what protects the agency in the month the number falls.

Write the panel with the client, and freeze it before the first fix

Each client gets one panel of its own. The client knows how its buyers ask, and the agency knows how to turn that into a panel, so the questions start from the client's own record: the first lines of inquiry emails, what the sales team hears on first calls, the reasons written down when a deal was lost. The agency shapes them into rows and a file, as setting up prompt monitoring describes, and the client reads every prompt before it is run.

Then both sides sign off on five things: the questions, the assistants, the number of runs, the verdict rules with their weights, and the date of the baseline. They belong in the statement of work or an appendix to it. Our methodology treats the panel and the yardstick as frozen once an engagement starts, and that is the standard to hold a client's panel to.

Run the baseline before anything ships. If the first fix goes live before the first reading, the baseline already contains the agency's work, and whatever that fix did has vanished from the comparison. A week between the baseline and the first change costs little and keeps the before honest.

After sign-off, the agency never edits a question on its own. A line that turns out to be phrased the way no buyer talks is retired with the client's agreement, a note and a date, and anything added in its place is reported apart until it has a baseline of its own. Folding a new line into the old total is how a flattering month gets built without anyone meaning to build it.

What goes with every monthly report to a client

The page a client reads can be short; GEO reporting for management sets out a one-page layout. What cannot be short is what travels with it. An agency sends four attachments every month, in a good month and a bad one.

• The panel as signed, with the date it was frozen, and every line retired or added since, each with its date.
• Every answer, verbatim, for every run on every assistant, with the date, the assistant, the run number and the verdicts marked on it. Not a selection, and not only the answers that named the client.
• The change log: every change to the client's site and listings that could move an answer, with the day it went live and who made it, whether the agency, the client's own staff, another vendor or the host.
• The spread between runs for the month, per assistant, printed next to any number the report calls a change.

The change log has to hold changes the agency did not make. A redesign by the client's web team, a new booking plugin, a price change from the sales director: any of them can move an answer in the same month as the agency's fix. A log that lists only the agency's work turns every rise into the agency's rise. Tracking AI visibility gives the log's fields; the agency's job is to fill them for everyone.

What does run-to-run variation mean for a month-on-month number?

It means the number can move when nothing has changed. Put the same question to the same assistant twice and the second answer can name a different set of businesses. An agency panel can run every question three times on each of ChatGPT, Claude, Gemini and Perplexity and average the runs; our own method asks each question once per reading of Claude and reads the pattern across twelve questions, and with monitoring across months. Either way the monthly visibility score is a sample, and two samples of an unchanged situation can differ.

Hedged answers are part of the variation. In the probe from our audit of 9,483 small business websites, one question per business went to Claude with web retrieval on, and 5,928 of the 9,240 answers gave a clearly differentiated list; the others hedged. An answer that hedges may name no business at all, so a month with more hedging can pull a client's named rate down with nothing different on the site.

So the report states the spread and judges every change against it. A move smaller than the gap between the month's lowest and highest runs goes in the report as exactly that: "within the range between runs; no change we can report". A larger move on one assistant in one month is a candidate, marked as one. How to tell a real change from noise is set out in measuring AI visibility; what an agency adds is the discipline of printing the range beside the claim, on every page, including the months it spoils a good story.

Report the four assistants apart as well. A total across four can rise while one of them falls, and the one that fell is what a client would want to hear about first.

Claims a reading supports, and claims it does not

The record can show what changed on the site, when, and what the answers said before and after. It cannot show on its own that one caused the other. A competitor may have published in the same month, the client's staff may have changed something the log missed, and an assistant can change between two readings with no notice to anyone outside its operator.

Our methodology page lists attributing impact, by showing which measure won which questions, among the things a score can do. For one client in one month, that is an order of events stated with its dates, not a proof. Label a judgment as one: "we think the new page is the likeliest reason" is a sentence the record can stand behind. "The new page caused it" is not.

Who owns the panel when the client leaves?

The client should, and the contract needs to say so before the first reading: it is the question nobody wants to raise at the end. The client paid for the series, and a series is only worth something if it can outlast the agency.

Hand over, complete and in files that open without the agency's tools: the panel file as signed, with every retired and added line; every answer verbatim with its date, assistant and run number; the verdict rules and weights, written out well enough for someone else to apply; the change log; and the monthly reports. A client who leaves with the reports alone has the verdicts and none of the evidence behind them.

The next agency can extend the series only by keeping the same questions, assistants, runs and weights. If it changes any of them, it has started a new series, and its first report should say so rather than set its first month beside the old agency's last. An agency whose weights are private leaves the client a series nobody else can extend, and that is worth saying before the contract is signed.

Some instruments were the client's all along. Search Console's generative AI performance report, rolled out to all websites on August 31, 2026, shows impressions of a site's links in AI Overviews and AI Mode by page, country, date and device, with no query dimension. The question staff ask new customers about how they found the business is the client's record too. Keep both in the client's own accounts, with the agency given access rather than ownership, so they stay where they are when the agency goes.

For the agency's side of this, AI visibility for marketing agencies covers the panel an agency runs on itself, the page for marketing agencies covers what its own site states, and selling GEO as an agency service covers what the contract promises.