How to Read AI SEO Tool Reviews Critically

Most AI SEO tool reviews read like product tours. They list features, quote the vendor's headline numbers, and award stars. That tells you what the tool says it does. It doesn't tell you whether its numbers mean anything.

The fix is a single question, asked of every sentence that contains a number or a promise: what measurement produced this, and could I reproduce it?

• If the reviewer names the data source, the prompts, the engines and the date, the claim is checkable.
• If the number comes from the vendor's own dashboard with no method behind it, it's marketing copy that happens to appear in a review.
• If the number describes something no engine publishes, nobody could have measured it, including the vendor.

That last category is bigger than most readers expect. The rest of this guide explains why, which claims fall into it, and what a useful review checks instead.

AI answers pass through four stages, and each one limits what any tool can observe.

1. Parametric: what the model knows with no web access, frozen at a training cutoff. Nobody changes this in the short term.
2. Access: whether a crawler may fetch and store your page at all. You and your CDN control this.
3. Retrieval: whether your page enters the model's context for a given question. The engine decides.
4. Generation: which retrieved pages get used and cited. The model decides.

Tools see the outside of stages 3 and 4, and only partly. As of October 2026, retrieval is discretionary on every major platform. ChatGPT decides per query whether to search, and OpenAI says it rewrites the user's prompt into one or more targeted queries before searching. Google may fan a question out into several related searches. So the prompts a tracker types are a proxy for queries nobody outside the engine can see.

Position data doesn't exist either. Google's Search Console report for AI Overviews and AI Mode shows impressions only, with no clicks, no click-through rate and no position. And citation isn't the same as use: OpenAI's API returns the full list of sources a model consulted separately from the citations, and the source list is often longer.

The research matches this picture. A July 2026 critical survey of 45 GEO studies (arXiv:2607.14035) concluded that already-retrieved content can alter its own citation, but no reviewed technique shows a stable, cross-platform causal effect on discoverability. Our digest of the critical survey of 45 GEO studies walks through what that means.

Three kinds of claim show up in reviews constantly, and none of them can be verified. When a review repeats one without challenge, discount the rest of it.

The llms.txt claim deserves its own read: see whether any engine reads llms.txt.

One more flag: a single blended "AI visibility score" with no breakdown. Google says AI Overviews and AI Mode may use different models and techniques, so even one company's surfaces don't share a denominator. A blended score without a per-engine split hides more than it shows.

The claims that hold up are mostly access claims. They're mechanical, the engine operators document them, and a reviewer can check them in an afternoon.

Per-bot crawler rules

robots.txt rules apply per bot. OpenAI documents that blocking GPTBot stops training use, while OAI-SearchBot controls ChatGPT search, and that each setting is independent. A tool that treats "AI bots" as one switch, or tells you to allow GPTBot so ChatGPT can find you, gets this wrong. A review should check whether the tool reports each bot separately.

Firewall and CDN reachability

A firewall or CDN can block an AI crawler that robots.txt allows. OpenAI's own eligibility wording requires both: allow OAI-SearchBot and confirm your host or CDN accepts its traffic. A tool worth reviewing requests pages as each crawler and reports the status code. Our playbook on reviewing CDN bot settings for AI crawlers shows what to look for.

JavaScript rendering

Vercel found that ChatGPT and Claude crawlers fetch JavaScript files but don't execute them, so content that appears only after scripts run may be invisible to them. A credible tool diffs raw HTML against the rendered page. If your site is a single-page app, read whether ChatGPT can read a React website.

Snippet controls

noindex, nosnippet, max-snippet:0 and data-nosnippet keep content out of AI answers on Google and Bing. Templates often apply them by accident. A tool should flag them.

Factual accuracy, per engine

AI platforms get specific facts wrong about specific brands. A review can check whether the tool shows which engine said what, with the prompt, the date and whether web search was on.

Reviews often blur three groups of things. Separating them is the fastest way to spot an overpromise.

What you control: crawler access, server-side rendering of the content that matters, snippet directives, and the facts on your own pages. Stating a specific fact (a price, a number, a one-line definition) on a page crawlers can fetch gives an engine a sentence it can quote. That's the only positive experimental signal for on-page content.

What a tool controls: how honestly it measures. It chooses the prompts, the engines, the retrieval state and how it combines results. Any single AI visibility score, including 99Visibility's, averages answers given with and without live retrieval. A good tool says so in its own documentation.

What nobody outside the engine controls: the model's training data, whether it searches for a given query, which retrieved page it cites, and how prominent your competitors already are. Brand prominence shows a strong association with citation in the evidence, with no lever attached. When engines cite third-party sites instead of yours, being named on those sites is the route in. See whether affiliate review sites decide which brands AI recommends.

So when a review credits a tool with "getting you cited" or "ranking you in ChatGPT", it's crediting the tool with something no tool controls. Treat it as the vendor's claim repeated, not as a test result.

Use this sequence on any review, or any vendor page, before you trial a tool.

1. List every number. Pull out each percentage, score, rank and count the review mentions.
2. Trace each one. Mark it as measured by the reviewer, quoted from the vendor, or unsourced. Only the first kind is evidence.
3. Check the method. Look for the prompt list, the engines, the dates, the sample size and whether search was on. Missing method means the number can't be compared to anything.
4. Look for the per-engine split. If results arrive as one blended score, ask for the breakdown.
5. Check for access tests. A tool that never checks robots.txt per bot, firewall responses or rendering is skipping the levers with the strongest evidence.
6. Note the incentive. Vendors and affiliates have one. That doesn't make their data wrong. It's why method disclosure matters.
7. Take your own snapshot. Run a few buyer questions yourself. The free checker on this site shows whether engines name you for up to three buyer questions. It's a single snapshot, not tracking, so treat it as a baseline to compare a paid tool's output against.

If you buy, repeat the comparison on a schedule. The quarterly AI visibility review checklist gives a structure for it.

• Reading a trend across changed methods. Ahrefs found 38% of AI Overview citations came from pages in the top 10, using its March 2026 method. An earlier, higher figure used different parsing, so the drop between them isn't a measured trend. Reviews often present it as one.
• Treating citation counts as influence. Retrieved and cited are different sets, and citation counts are a noisy proxy for whether a page shaped the answer.
• Assuming a prompt tracker sees everything. In Otterly's 2026 experiment, Microsoft's own AI Performance report counted 42,817 citations for one test site while Otterly's prompt-based tracker logged 3,934. The windows may differ, but any prompt set is a sample.
• Accepting blended results. In the same experiment, 2,718 of the 3,934 tracked citations came from Microsoft Copilot, while ChatGPT, Gemini, AI Overviews and AI Mode almost never cited the site. One averaged score would have hidden that.
• Overcorrecting. A review that says "schema is harmful, remove it" overreaches too. Google says existing markup does no harm. The honest position is that schema isn't a lever for AI answers.
• Confusing training and search bots. Blocking GPTBot doesn't remove you from ChatGPT search. Reviews that say otherwise didn't read OpenAI's crawler documentation.

Are AI SEO tools worth paying for at all? They can be, if you buy them for what they can measure. A useful tool checks crawler access per bot, firewall responses, JavaScript rendering and snippet directives, and shows which engine said what about your brand. Those findings lead to specific fixes. A tool sold mainly on rank tracking, uplift promises or authority scores is selling numbers that no engine exposes, so the subscription buys reports you can't act on or verify.

Can any tool show my rank in ChatGPT or Google AI Overviews? No tool can show a verified rank. As of October 2026, Google's Search Console report for AI features shows impressions only, with no position data, and OpenAI says only that ChatGPT ranks results using multiple factors. A tool can report whether you appeared in answers to its own prompts on a given day. That's presence in a sample, not a position, and a good tool labels it that way.

Is a study run by a tool vendor automatically untrustworthy? No. Vendor incentive is a reason to check the method, not a reason to dismiss the data. Look for a named control group, a published prompt list, the engines and dates covered, and a sample size. Matched-control designs, like the Ahrefs schema test, are stronger than before-and-after anecdotes. Be most skeptical when a vendor's study happens to prove that the vendor's own feature works.

What should an honest AI visibility report include? It should report each engine separately, state whether answers were generated with web search on or off, name the prompts and dates, and separate measurements from the tool's own modelling. It should flag access problems such as blocked crawlers or content that only appears after JavaScript runs. It should never promise a score, a ranking or citations, because no tool controls what the models say.

Do star ratings on software review sites help me choose? They tell you about usability, support and billing, which matter. They rarely tell you whether the tool's numbers are valid, because most reviewers can't test that. Use star ratings to shortlist, then apply the measurability test yourself: trace each headline number to its method, ask for the per-engine breakdown, and compare the tool's output against a snapshot you ran independently.