Should an AI visibility tool show the full AI answers, not just a score?

A visibility score compresses many answers into one number. That's useful for a trend line. It's useless for deciding what to change.

The core problem is retrieval. On every major platform, live web search fires per query and at the engine's discretion. ChatGPT decides when to search, Claude.ai search is a user toggle, and Google AI Overviews often don't appear at all. So any single score averages an unobserved mix of answers drawn from live pages and answers drawn from the model's frozen training data.

Two answers can score the same and be completely different problems:

• One answer never mentions you.
• Another names you but quotes last year's price.

The score can't separate them. The text can. And the sources tell you where the fix lives. If the engine cited a directory listing with your old price, the fix is on that directory. If it cited your own pricing page and still got it wrong, check whether the price exists in the HTML before JavaScript runs. That's why a tool built to fix rather than only track has to show you the answers.

Use this as a buying checklist. For each prompt and each engine, the tool should show:

1. The exact prompt sent. Not a paraphrase. Small wording changes move answers.
2. Engine, model and date. "ChatGPT" is not one thing. OpenAI's non-reasoning Instant path carries an August 31, 2025 knowledge cutoff, while the GPT-5.6 family carries February 16, 2026.
3. Retrieval state. Was web search on or off for this answer? If the tool doesn't know, it should say so.
4. The full answer text. Unedited, with your brand mentions findable.
5. Cited sources with URLs. Which pages the engine credited.
6. Retrieved sources, where the API exposes them. OpenAI's API documentation states that the number of sources consulted is often greater than the number of citations. The gap is diagnostic.
7. Flagged facts tied to a sentence. Each "wrong fact" flag should point at the sentence and the fact on record it contradicts.
8. Failed runs marked as failed. An engine that errored returned no evidence. It shouldn't count as a clean or a bad answer. See how failed responses should affect a score.
9. Export. You should be able to download the raw log.

If a vendor can't show items 1 to 5, ask what its number is measuring. The broader questions to ask are in what a tool should disclose about its methodology.

Several documented facts make a single number lossy.

Surfaces differ

Google's AI features documentation says AI Overviews and AI Mode "may use different models and techniques" and can return different links. Averaging them blends two products.

Citation units differ

Anthropic's web search tool cites short spans of text, while Perplexity cites whole sources by index. A "citation count" across both adds unlike things.

One engine can dominate

In OtterlyAI's 2026 experiment, Microsoft Copilot produced 2,718 of the 3,934 tracked citations for one test site, while ChatGPT, Gemini, AI Overviews and AI Mode almost never cited it. A blended score would have hidden that split.

A prompt sample is a sample

In the same experiment, Microsoft's own AI Performance report credited the site with 42,817 citations, against 3,934 logged by Otterly's prompt tracker. The windows may not match, but it shows a tracker sees only the prompts it asks.

Cited is not the same as used

A mechanistic study of citation in retrieval-augmented generation (arXiv:2606.28358) repaired over 90% of missed citations by adjusting internal model components, without changing the sources. Citation counts are a noisy proxy for influence. Reading the answer shows whether your facts shaped it.

Raw answers are better evidence than a score, but they need context to read correctly.

• Answers vary between runs. The same prompt can produce different text. Look for patterns across runs, not one quote.
• Engines rewrite your prompt. OpenAI's help documentation says ChatGPT search typically rewrites a query into one or more targeted queries. The prompt the tool sent is not the query that retrieved pages. Bing Webmaster Tools is the one surface that shows real grounding queries, and as of October 2026 it has no API for them.
• Logged-in state changes results. OpenAI notes ChatGPT may use saved memories when rewriting a search query. Reproducible logs should come from logged-out or memory-off sessions.
• No citations means unknown grounding. If an answer shows no sources, you can't tell whether it came from live search or training data.
• Scoring formulas change. If a vendor revises how it scores accuracy, old and new scores aren't comparable. The answer log is what stays comparable.

Work from the text, not the number.

1. Read each answer for wrong facts. Note the price, product name, location or claim that's wrong, and which engine said it. Our guide to AI hallucinations about brands covers the usual causes.
2. Trace the cited source. If it's a third-party page, the route in is getting that page corrected. If it's yours, check that the right fact is stated plainly on a page that can be fetched.
3. Check the fact exists before JavaScript runs. Most AI crawlers other than Googlebot don't execute JavaScript. A raw HTML vs rendered DOM check shows what they can read.
4. Check access per bot. Confirm robots.txt allows OAI-SearchBot, PerplexityBot and Claude-SearchBot, and that your firewall or CDN doesn't return 403 to them.
5. Re-run the same prompts later. Compare the text, not just the score.

To see raw answers before buying anything, the free checker on this site runs up to three buyer questions and shows what came back. None of these steps guarantees a citation. They remove the blocks you control.

Is an AI visibility score useless? No. A score is a reasonable trend line if the method stays fixed and the vendor discloses what it averages. The problem is acting on it alone. It mixes engines, surfaces and answers with and without live search. Use the score to notice change, then open the answer log to see what changed and why.

What is the difference between retrieved and cited sources? Retrieved sources are the pages an engine pulled in while building an answer. Cited sources are the ones it credited in the reply. OpenAI's API documentation says the retrieved set is often larger. A page that's retrieved but not cited reached the engine and lost out at the answer stage. A page never retrieved points to an access, index or relevance problem.

Can I trust AI answer logs from a logged-in account? Treat them with caution. OpenAI says ChatGPT may use saved memories when rewriting search queries, and custom instructions can shape answers. That makes results hard to reproduce or compare. For evidence you can repeat, use logged-out or memory-off sessions, or API calls with retrieval state recorded.

Does seeing the full answer tell me how to rank higher in AI search? No. Nothing published shows a reliable way to rank inside AI answers, and Google exposes no position data for its AI features. The answer log tells you what an engine said and which sources it used. That shows wrong facts to correct and access problems to fix. It doesn't promise a citation.