Why do AI visibility tools give different results for the same brand?

An AI visibility score isn't read off a dashboard that ChatGPT or Google publishes. A tool builds it. The tool picks prompts, sends them to some models, records the answers, and turns them into a number with its own formula.

Every one of those steps is a choice. Change the choice and the number changes, even when your brand hasn't changed at all.

There's also no referee. Google's generative AI report in Search Console exposes impressions only, with no clicks, CTR or position. OpenAI and Perplexity publish no brand-level visibility count. So when two tools disagree, there's no official figure to show which one is closer.

What this means for you: a gap between two tools is normal and usually says more about method than about your brand. The next move is to find out which settings differ before reading anything into the gap.

Prompt set

Tools test prompts they chose, or prompts you typed. The engines don't search with those prompts. OpenAI says ChatGPT search typically rewrites a query into one or more targeted queries before it searches. Google may fan a query out into several related searches. Any prompt set is a proxy for an input nobody outside the engine can see.

Model

"ChatGPT" isn't one model. OpenAI documents a February 16, 2026 knowledge cutoff for the GPT-5.6 family and an August 31, 2025 cutoff for the Instant path. Many tools call an API model that may not match what the consumer app serves. Our page on whether tools test the same models people use covers this in detail.

Retrieval state

As of October 2026, retrieval is discretionary on every major platform. ChatGPT decides per query whether to search. Claude.ai search is a user toggle. Perplexity's Agent API needs web search switched on. A tool that runs with search off measures what the model remembers. A tool with search on measures what it finds today.

Location, language and session

AI Overviews don't appear on every results page and vary by location and language. Google also says AI Overviews and AI Mode may use different models and techniques, which is why a tool's "Google AI" result can differ from the AI Overview you see. ChatGPT may use saved memories when it rewrites a search query, so logged-in tests aren't reproducible.

Scoring formula

One tool counts mentions. Another counts citations. Another weights accuracy or sentiment. Units differ too: Anthropic's web search cites text spans, while Perplexity cites whole sources by index number. There's no shared denominator.

Three published cases show how far method alone moves a number.

• Prompt samples versus engine counts. In Otterly's 2026 experiment, Microsoft's own AI Performance report credited one test site with 42,817 citations, while Otterly's prompt-based tracker logged 3,934. The time windows may not match, so it isn't an exact ratio. It does show that a prompt sample sees a fraction of what the engine counts.
• Two studies, two overlap figures. Ahrefs found that 37.9% of URLs cited in AI Overviews also appeared in the top 10 results (March 2026). BrightEdge reported 16.7% from the top 10 in September 2025, with no published sample size. The methods differ, so the numbers can't be reconciled.
• Formula changes. Under 99Visibility's original accuracy formula, the same 10% error rate scored 90 at 20 answers and 0 at 200 answers, because penalties had no denominator. The revised formula scores both at about 88. Same answers, different maths, different score. This is also why old and new scores from one tool may not be comparable.

The takeaway: method choices can produce gaps bigger than any real change in your visibility.

The same tool disagrees with itself. Models sample answers, so the same prompt can return a different answer on a second run. See answer variability for why.

Blended scores hide the split. In Otterly's test, Microsoft Copilot produced 2,718 of one site's 3,934 tracked citations, while ChatGPT, Gemini and Google's AI features barely cited it. A single blended number would have hidden that. The trade-offs are laid out in one score versus per-platform scores.

Brand size skews benchmarks. Large brands dominate AI answers. A benchmark that compares you with much bigger competitors is partly reporting brand size.

Cited isn't the same as used. OpenAI says the number of sources the model consulted is often greater than the number it cites. Tools that count only citations undercount.

If you need to compare two tools, or two runs, match these first:

1. Identical prompts, word for word, and the same number of them.
2. The same engine and model, named precisely, not "ChatGPT" or "Google AI".
3. The same retrieval state: search on or search off, recorded per answer.
4. The same location and language.
5. Logged-out, memory-off sessions.
6. The same date window and number of runs per prompt.
7. The same metric definition: mention, citation or accuracy, and how each is counted.
8. A published formula, including weights and how failed answers are handled.

If a vendor can't tell you these, you can't compare its number with anything. Our guide to what AI visibility scores average explains what to ask for. Then compare platform by platform, and track one tool's trend over time rather than switching tools between readings.