Visibility on Perplexity: what its crawlers fetch, and what a page has to give them

How does Perplexity find and cite a page?

Perplexity is an answer engine: it writes a reply to the question and lists the pages it drew on, numbered, beside the sentences they support. That design puts the sources in front of the reader in a way a chat window does not, and it means the question for a business is concrete. Not "does Perplexity know us" but "is our page one of the ones it opens and cites for the questions our buyers ask".

Perplexity's crawler documentation describes the two ways a page gets in front of it. PerplexityBot crawls the web to build the index that answers are assembled from; it identifies itself by that token and reads robots.txt. Perplexity-User fetches a specific page when a user's request calls for it, and the documentation says it generally does not obey robots.txt, because a person asked for that page. The first is how you become findable for a question that does not name you; the second is what happens when someone pastes your URL or asks about you directly.

Does blocking PerplexityBot, or allowing it, change what you get cited for?

Allowing it is the precondition; it is not a lever. A robots.txt that disallows PerplexityBot keeps your pages out of the index that answers to a stranger's question draw on, which is the visibility a business wants. A robots.txt that allows it puts you in the pool with every other page that allows it, which is most of the web: in our audit of 9,483 small business websites (40 trades, 90 metros, September 2026), 3.2% disallowed AI crawlers on purpose.

The block that matters more is the one nobody wrote. 7.2% of the same sites answered our crawler with a challenge page from a CDN or a security plugin, which stops PerplexityBot's fetch as surely as a rule in the file and never appears in the file at all. A further 3.0% served their text only through JavaScript, and a fetcher that runs no scripts received an empty shell. Which tokens to name and how the fallback group works is in robots.txt for AI crawlers; the crawler access check reads your file for PerplexityBot and the other eighteen in a few seconds.

One consequence of the user-fetch token is worth stating plainly. Because Perplexity-User generally ignores robots.txt, a page you have blocked from the index can still be read when a user asks about it by name, and Perplexity says so. A block is not privacy. It only decides whether you are found by strangers.

What a page has to state to be worth citing

An answer engine cites the passage that answers the question, so the page has to contain one. The pattern is the same one every article on this site describes, because every retrieval system needs it: the question a buyer asks as a heading, forty to eighty words directly under it that answer it completely, then the detail. What Perplexity's design adds is exposure: the citation is visible, so a passage that is wrong or vague is wrong or vague in public, next to your name.

Nothing in that table is specific to Perplexity, and that is the finding. The same page that an answer engine can cite is the page an assistant with retrieval can quote and a search engine can snippet. The work is one set of pages, written to be lifted, with the facts in one form everywhere.

Freshness, dates and authorship

An engine that shows its sources is choosing among candidates it can display, and a page that says when it was written and by whom is a better candidate than one that says neither. State the date the page was published and the date it last changed, in the visible text and in the Article markup, and name the author with a page of their own. Keep the dates true: a modified date that moves on every deploy is a claim a crawler learns to ignore.

For a service page the same idea is simpler. State the year the business started, the license or credential by its plain name, and the fact that the prices on the page are current as of a stated month. An answer that cites you will often repeat those facts; make them the facts you would want repeated.

How to see what Perplexity cites for your buyers' questions

1. Write down ten questions your buyers ask, the way they type them: the problem question, the comparison, the one that names you. This is the same panel you would use for any assistant.
2. Ask each one on Perplexity three times, on different days, and record which pages it cites, in order, and whether one of them is yours. One run is a coin flip; the pattern across runs is the finding.
3. Open the pages it cited instead of yours and read what they state that yours does not: an answer under the question, a price, an area, a date, a review.
4. Fetch your own page as a crawler would and read what comes back. If the answer you wrote is not in that text, it is not in what PerplexityBot indexed.
5. Fix the page, then measure again next month with the same ten questions. Our own measurement does not cover Perplexity: it puts twelve questions to one assistant, Claude, once per reading, and keeps every answer word for word with the pages it read, so a citation can be checked later.

What this article does not claim

We have not measured how often Perplexity cites small business pages, and we will not borrow a figure that says we did. The probe in our audit used Claude, not Perplexity, and its 14.9% naming rate is a Claude number for one question per business on one day. What we can say about Perplexity is what its documentation says about its crawlers, and what our crawl found on the pages those crawlers would fetch. The rest is a measurement you can take yourself in an afternoon; ours is taken with Claude, not Perplexity.