Is every AI claim a tool flags as wrong actually false?

An accuracy flag tells you an AI answer did not line up with what the tool knows about your brand. It does not tell you the answer is false. There are two very different reasons a claim gets flagged.

• Contradicted. You told the tool your Starter plan costs a certain amount, and the AI stated a different price. The claim conflicts with a fact on record. Treat it as an error.
• Unverifiable. The AI stated your employee count, and you never gave the tool an employee figure. Nothing on record supports or refutes it. The claim might be right, wrong, or out of date.

Only the first is a confirmed false claim. The second is an unverifiable claim: a question about your fact list as much as about the AI. Some tools label both as hallucinations and weight them the same, so the raw flag count overstates how wrong the AI really is.

The practical move: sort every flag into one of these two buckets before you spend time correcting anything.

The blur is not a tool bug. It comes from how factuality research defines "supported."

FActScore, the atomic-fact metric from Min et al. (EMNLP 2023), scores a response as the share of its facts supported by a knowledge source. Its "not supported" class covers a source that contradicts a claim or lacks evidence for it. The metric does not separate the two. RAGAS faithfulness works the same way: an unsupported claim counts against the answer whether it is wrong or simply unchecked.

Other benchmarks do carve out a neutral bucket, but for a different situation:

• SAFE (Wei et al., NeurIPS 2024) has an "irrelevant" class that drops a claim from scoring.
• Vectara's hallucination leaderboard excludes refused responses from its denominator, so refusals cannot pass as accuracy.

Both exclusions cover a model that declines to assert. Brand monitoring faces the opposite case: the AI asserts something confidently, and the checker cannot confirm it. That sits closer to FActScore's "not supported" than to an abstention.

So no benchmark settles whether an unverifiable brand claim should count as an error. Splitting it out is a product choice. FActScore also gives every fact equal weight, so any tool that grades a wrong acquisition as worse than a wrong headcount is making its own judgment call, not following a standard. Ask your tool how it treats both.

A checker can only catch contradictions for facts it holds. Every fact you leave out becomes a place where the AI can say anything and the tool can only shrug.

Say you run a hypothetical accounting software company and you supply only pricing and founding year. An AI answer that names your HQ city, your integrations, and your CEO produces three unverifiable flags. Add those three facts and each flag resolves: it becomes a confirmed match (no action) or a confirmed contradiction (worth fixing).

The facts that resolve the most flags are usually the ones AI answers state most often:

• Pricing tiers and what each includes
• Founding year, HQ, and ownership (including acquisitions)
• Product names, current and retired
• Key integrations and certifications
• Founders and executives (see our guide to checking what AI says about your founders)
• Headcount, if you are willing to state it

Keep the list current. A stale fact on record turns a correct AI answer into a false positive. If you changed pricing last month and did not update the list, the tool will flag the AI for being right.

Even a genuine contradiction does not always mean the model invented something. Check these before you treat it as a hallucination.

Your own site disagrees with itself

If your pricing page, about page, and footer state a fact three different ways, the AI may be quoting one of them accurately. Fix the inconsistency on your site first. That is fully in your control.

The answer came from the model's memory

Retrieval is per query and discretionary on every major platform. An answer given without live search draws on training data frozen at a cutoff, so it can repeat a fact that was true a year ago. Your current pages cannot reach that answer.

A third-party page said it

The AI may be faithfully repeating an outdated directory listing or review site. Learning how to find which source caused a wrong fact tells you where the correction belongs. Note that citations do not always match the claim they sit next to, which is why AI citations sometimes do not support the claim.

The wording is close but not the same

"Founded in the early 2010s" against a fact of 2012 is vague, not false. A specific wrong number is worse than a vague true one.

Work through each flag in this order.

1. Find the fact it was checked against. No fact on record means the flag is unverifiable. Decide whether to add the fact, then re-check.
2. Confirm the fact on record is current. If it is stale, update it and dismiss the flag.
3. Read the AI's exact wording. Rule out paraphrase, rounding, and vague-but-true phrasing.
4. Note the platform and whether the answer cited sources. Visible citations suggest live retrieval. No citations may mean a training-data answer you cannot change this quarter.
5. Check your own pages. Make sure the correct fact appears plainly in the HTML, stated the same way everywhere. A specific fact on a fetchable page gives an engine a sentence it can quote.
6. Prioritize by damage. A wrong price or a false acquisition matters more than a wrong headcount.

For a wider view of causes, see AI hallucinations about brands, and for the mechanics behind detection, how AI visibility tools detect factual errors. If you want a quick snapshot before setting up a full fact list, the free checker on this site shows how AI answers a few buyer questions about you.