AI Visibility Tools: What They Measure, and What They Can't

Most AI visibility software reports some mix of four numbers. All of them are computed on prompts that you or the vendor wrote.

• Mentions. Did the answer name your brand? Usually shown as a rate across the prompt set.
• Citations. Which URLs the engine linked as sources, and whether any are yours.
• Sentiment. Whether the description reads positive, neutral or negative. Another model usually makes that call.
• Accuracy. Some tools compare the answer against facts you supply, such as pricing or founding year, and flag what's wrong.

Share of voice is typically your mentions divided by mentions of you plus named competitors, over the same prompts. Measuring AI share of voice walks through the arithmetic.

What it means: every one of these numbers is a property of the sample. A tool can't see what real users typed, which answers they got, or what they did next. Treat the prompt list as the most important setting in any tool, because it is.

The prompt is not the query

OpenAI's help page on ChatGPT search says it typically rewrites your query into one or more targeted queries that it sends to search providers. Google may fan a question out into several related searches across subtopics. The Gemini API generates its own search queries. So tracking the exact words a buyer might type measures a proxy. As of October 2026, the only place that shows the queries an AI actually ran is the AI Performance report in Bing Webmaster Tools, and it has no API.

Retrieval state changes the answer

Retrieval is per query and discretionary. ChatGPT decides when to search. Search in Claude.ai is a user toggle. Perplexity's Agent API needs web search switched on. With search off, the answer comes from training data frozen at a cutoff: OpenAI lists Feb 16, 2026 for the GPT-5.6 family and Aug 31, 2025 for the Instant path. Published estimates of how often ChatGPT searches run from about 18% of conversations (Profound) to 80.5% of a prompt set (cloro), because they measure different things. A score that blends both states averages two different systems.

API is not app

Many tools call vendor APIs. The consumer app may serve a different model, and memory or custom instructions shape what a logged-in user sees. OpenAI says ChatGPT may use saved memories when rewriting a search query. Ask which model and surface produced each number.

Cited is not the same as used

OpenAI's API returns the sources a model consulted separately from its citations, and notes that sources often outnumber citations. Research on attribution (arXiv 2606.28358) repaired over 90% of missed citations by adjusting internal model components, without changing the sources. Platforms also cite in different units: Anthropic cites text spans, Perplexity cites whole sources by index. Claude's web search tool docs show what that looks like in practice.

As of October 2026, Google's generative AI report in Search Console shows impressions for AI Overviews and AI Mode. It shows no clicks, no click-through rate and no position. Google's page on third-party SEO tools says those tools don't have access to its internal ranking data. OpenAI's ranking disclosure is one sentence: ChatGPT ranks results using multiple factors, and placement is not guaranteed. Any "AI rank" is a vendor construct with nothing to check it against.

Google also says AI Overviews and AI Mode may use different models and techniques. Two Google surfaces can disagree about the same question.

Sample size is the other limit. In Otterly's 2026 experiment, its prompt-based tracker logged 3,934 citations for one test site, while Microsoft's own AI Performance report credited that site with 42,817. The time windows may not match, so it isn't an exact ratio. It does show that a prompt sample sees a fraction of what the engine sees. Analytics doesn't close the gap either, as this note on Google Analytics and AI visibility explains.

The next move: report presence, citations and accuracy per engine, with retrieval state recorded. Drop position from your reporting.

An AI answer passes through four stages. A tool sees the output of all four mixed together.

In your control:

• Per-bot robots.txt rules. OpenAI's crawler page says each setting is independent. GPTBot governs training. OAI-SearchBot governs ChatGPT search.
• Firewall and CDN reachability. A firewall can return 403 to a crawler that robots.txt allows.
• Server-side rendering. Vercel found ChatGPT and Claude crawlers fetch JavaScript files but don't execute them.
• Index and snippet eligibility. Google says a page must be indexed and eligible for a snippet to appear as a supporting link in AI Overviews or AI Mode. noindex, nosnippet, max-snippet and data-nosnippet keep content out.
• Stating specific facts (a price, a number, a one-line definition) on a page that can be fetched.

Not in your control: query rewriting, whether retrieval fires, where your page lands in the context, and what the model learned in training. A 2026 critical survey of 45 GEO studies (arXiv 2607.14035) found that already-retrieved content can change its citation, but no reviewed technique showed a stable, cross-platform effect on discoverability. A tool that sells a list of content tweaks is selling past the evidence.

Use these with any vendor. Good tools answer them without hedging. What a tool should disclose about its methodology covers the full list.

1. Which prompts? Who wrote them, how many, and can I export them?
2. Which surface? API or consumer app, and which model version, for each engine?
3. Was search on? Is retrieval state recorded per answer, or blended?
4. Which session? Logged out, memory off? If not, results aren't reproducible.
5. How many runs? Answers vary between runs. How does the tool handle that?
6. Retrieved or cited? Does it count only citations, or also the sources consulted?
7. How is the score built? What are the weights, and which parts are measurements versus the vendor's model?
8. Does it check access? Does it test crawler reachability and rendering, or only read answers?
9. What do recommendations rest on? If the advice is schema, llms.txt, an E-E-A-T score or domain authority, ask for the primary source.

If you're comparing paid options, which GEO tools are worth paying for applies these questions to the market.

Fix the gates before you measure what's behind them.

1. Check crawler access. Confirm robots.txt lets OAI-SearchBot, PerplexityBot and Claude-SearchBot in. Then confirm your firewall does too. 99Visibility's AI Crawler Checker requests a page as each AI crawler and shows which ones get turned away.
2. Check rendering. Compare raw HTML with the rendered page. Anything that appears only after JavaScript runs, such as pricing, may be invisible to most AI crawlers.
3. Check index status per engine. Look at Google Search Console and Bing Webmaster Tools separately. In Otterly's experiment, Google deindexed both test sites and its AI surfaces stopped citing them, while Bing kept them and Copilot kept citing.
4. Write and freeze a prompt set. Use real buyer questions. Record the wording, engines and settings. Setting an AI visibility baseline covers this step.
5. Run a baseline per engine. Record retrieval state for each answer. For a quick first look, the free checker on this site runs up to three buyer questions and scores the answers. It's a snapshot, not tracking.
6. Fix wrong facts. Where an answer gets your price or product wrong, state the correct fact plainly on a fetchable page.
7. Re-measure on the same terms. Same prompts, same engines, same settings. Expect run-to-run noise, and don't read one movement as a trend.

• Trusting a blended score. One number across engines and retrieval states hides the detail. In Otterly's test, Copilot produced 69% of one site's tracked citations while ChatGPT, Gemini and Google's AI surfaces barely cited it. A blend would have hidden that.
• Comparing scores across tools. Different prompts, models and formulas produce different numbers. Even one tool's score can change meaning when its formula changes.
• Ignoring brand size. Big brands dominate AI answers. Semrush found only 36 brands visible across all four surfaces it tracked every month, including YouTube, Google, Reddit and Amazon. Benchmark against similar-sized competitors.
• Allowing GPTBot to get into ChatGPT search. GPTBot governs training only. OAI-SearchBot controls search.
• Acting on markup advice. Google says structured data isn't required for generative AI search, and Ahrefs' matched test found AI Overviews citations fell 4.6% after pages added schema. Google says Search ignores llms.txt. That doesn't mean you should rip out existing markup. It means markup isn't the lever.
• Tracking only your own mentions. Answers often cite third-party sites. AI brand monitoring covers what to watch beyond your domain.

Are free AI visibility tools good enough to start with? For a first look, yes. A free check on a few buyer questions shows whether engines name you and whether the facts are right. It can't show trends, because a handful of prompts run once is too small to separate change from noise. Use it to find obvious problems, like a wrong price or a missing brand, then fix access issues before deciding whether continuous tracking is worth paying for.

How often should I re-run an AI visibility check? Re-run after a change you made, such as fixing crawler access or correcting a fact, and on a steady schedule otherwise. Monthly suits most teams. Keep the prompts, engines and settings identical each time, or the comparison means nothing. Model updates and retrieval changes on the engine side can move results without anything changing on your site, so note launch dates when you read a shift.

Can an AI visibility tool tell me why I'm not cited? Only partly. A tool can check things you control: whether crawlers are allowed and reachable, whether content renders without JavaScript, and whether pages are indexable. It can't see retrieval or ranking inside the engine, so it can't prove why one page was chosen over another. Treat any explanation beyond access and factual accuracy as a hypothesis, not a finding.

Does a higher AI visibility score prove my work paid off? No. A score rises or falls with the prompt sample, the share of answers that used live search, and model updates you didn't make. It also says nothing about traffic, since Google's generative AI report shows impressions only. Look at what changed specifically: a crawler that now gets a 200, a fact now stated correctly, a citation that now appears on the same prompt and engine.

Should I test AI answers while logged in? Not for measurement. Logged-in sessions carry memory and custom instructions, and OpenAI says ChatGPT may use saved memories when rewriting a search query. Your results then reflect your own history, not a typical buyer's. Test logged out with memory off, or through an API with settings recorded, so the same prompt gives a comparable answer next month.