How do AI visibility tools detect factual errors about a company?

Error detection is a comparison between what an AI engine says and what you know to be true. Most tools run the same four steps.

1. Collect answers. The tool asks each engine a set of buyer questions and stores every answer that comes back.
2. Extract claims. It splits each answer into single statements: a price, a founding year, a headquarters city, a product name, an owner.
3. Match against a fact sheet. Each claim is checked against verified brand facts. The result is one of three: supported, contradicted, or no fact on record.
4. Score. Contradictions are weighted by severity and divided by the number of answers that returned.

The claim-splitting step comes from research on factual precision. FActScore breaks long text into atomic facts and reports the share that a knowledge source supports. Brand error detection applies the same idea, with your fact sheet as the knowledge source.

What this means for you: the tool cannot catch an error about a fact it was never given. A thin fact sheet produces a clean-looking score that says little.

Every established factuality metric divides by a count of observations. FActScore divides by atomic facts in the response. RAGAS faithfulness divides supported claims by total claims. The Vectara hallucination leaderboard reports a hallucination rate across documents summarised and keeps refused responses out of the count.

A tool that sums penalty points without a denominator breaks as the sample grows. Say two audits both have a 10% error rate, one over 20 answers and one over 200. They should score the same. With raw penalties, the bigger audit scores far worse, so buying more questions makes your brand look less accurate.

A pure average has the opposite risk. The SAFE paper on long-form factuality introduced a recall term because precision alone "does not penalize systems that abstain from answering too often or generate text with fewer facts." In brand terms, one serious error (a wrong acquisition, a wrong price) can fade to nothing inside hundreds of clean answers. The 99Visibility score handles this by combining the error rate with a separate term for the single worst error, so a critical mistake stays visible at any sample size.

Two checks to run on any tool: does it count only answers that returned, and does one severe error still show up in a large audit?

When an AI states something your fact sheet does not cover, the tool has no basis to call it wrong. Say an engine gives your employee count and you never recorded one. That claim is unverifiable. It may be right.

Research metrics do not settle this for you. FActScore's "not supported" class covers claims the source contradicts or lacks evidence for, without separating the two. RAGAS also counts an unsupported claim against faithfulness. So a tool that treats both as errors is following the literature, but it is still mixing two different findings.

For a brand, the difference changes the fix. A contradiction needs a correction at the source. An unverifiable claim needs a fact added to your sheet, and then a recheck. Read more on whether every flagged claim is actually false.

Most tools rank errors as critical, moderate or minor. That is useful: a wrong owner matters more than a wrong headcount. It is not inherited from benchmarks. FActScore states that "every atomic fact has an equal weight of importance," and Vectara scores each summary as consistent or not.

So the weights, and any constants that shape the score curve, are the vendor's judgement. No public dataset backs them. A trustworthy tool says so. Ask any vendor which weights it uses and whether a minor unverifiable claim costs as much as a provably wrong price.

Several conditions make a flag less reliable than it looks.

• Your own facts disagree. If your pricing page, about page and footer state different numbers, the engine may be quoting you. See whether mistakes on your own website cause AI errors.
• Outdated, not invented. An answer given without live search draws on training data frozen at a cutoff. An old price there is stale, not hallucinated. The diagnosis and fix differ, as covered in hallucination vs outdated answers.
• Unknown retrieval state. As of October 2026, ChatGPT searches at the model's discretion and Claude.ai search is a user toggle. Unless citations appear, you cannot tell which kind of answer you got.
• Small samples. One flag in two answers reads as a collapse when it is really a partial run.
• Snapshots. An audit shows answers at one moment. Engines change, so real-time detection is a separate question.

1. Write a verified fact sheet. One line per fact: prices, plans, founding year, location, ownership, leadership, product names.
2. Make your site agree with it. State each fact the same way everywhere, in HTML that loads before JavaScript runs. Many AI crawlers do not execute JavaScript, per Vercel's crawler study.
3. Sort flags into contradicted and unverifiable. Fix the first, add facts for the second.
4. Trace each wrong fact to its source. It may be your page, a directory or an old article. Follow the plan for when AI gets your brand wrong, and for ChatGPT see trained knowledge vs search.
5. Recheck after fixes. The free checker on this site shows what engines say for up to three buyer questions. Nobody can force an engine to change its answer, so track the result rather than assume it.