AI SEO Tracking: What a Tracker Can't See, and How to Use It Anyway

An AI SEO tracker picks a list of prompts, sends them to each AI engine on a schedule, and stores the answers. From those answers it counts brand mentions and cited URLs, and some tools score sentiment or factual accuracy.

That is a sample. It is a sample of answers, to prompts the tool or you wrote, under conditions the tool chose (usually logged out, often through an API rather than the consumer app). It is useful for spotting change and catching errors. It is not a measurement of how buyers experience AI search.

The gap between the sample and reality has four parts, and none of them can be closed by a better dashboard. Knowing them tells you which numbers to trust and which to ignore. For the broader mechanics, see how AI visibility tracking works.

Real user queries

No tracker sees what your buyers type. Worse, the prompt is not what gets searched. OpenAI's help page on ChatGPT search says it "typically rewrites your query into one or more targeted queries" that it sends to search providers. Google may fan a query out into several related searches. The Gemini API generates its own search queries. So a tracker testing your chosen prompt is testing a proxy for an input it cannot observe. The one first-party window is the AI Performance report in Bing Webmaster Tools, which shows a sample of grounding queries for Copilot and Bing AI summaries, in the interface only. More on this in whether tools can see real AI prompts.

Retrieval state

As of October 2026, retrieval is discretionary on every major platform. ChatGPT decides per query whether to search. Claude.ai search is a user toggle with no published default. Google AI Overviews often do not trigger at all. If an answer shows no citations, nobody outside the engine knows whether it came from live search or from training data. Published estimates of how often ChatGPT searches disagree badly: Profound reports about 18% of conversations, Semrush reports 34.5% of US queries in February 2026, and cloro reports 80.5% on a 100-prompt set. They measure different things on different populations.

Training data is frozen at a cutoff. OpenAI lists February 16, 2026 for the GPT-5.6 family and August 31, 2025 for the Instant model behind chat-latest. An ungrounded answer can describe your company as it was a year ago.

Position

There is no rank inside an AI answer that anyone can verify. Google's generative AI report in Search Console shows impressions only, with no clicks, CTR or position. Google also states that third-party tools don't have access to its internal ranking data. Any "AI position" a tool reports is its own construct. See why AI has no rank to check.

Logged-in personalization

OpenAI says that if memory is enabled, ChatGPT may use saved memories when rewriting a search query, and that product results consider context such as memory and custom instructions. Location and language change what Google shows too. Trackers run clean sessions. Your buyers mostly don't.

Tracker numbers and engine numbers rarely match, and that is expected.

Otterly's 2026 experiment with two AI-generated blog sites shows the scale. Microsoft's own AI Performance report credited one site with 42,817 citations, while Otterly's prompt-based tracker logged 3,934 for that site. The time windows may not match, so it is not an exact ratio. It does show that a prompt sample sees a fraction of what the engine sees. Details in why Bing's citation counts differ from your tracker.

Three more reasons counts diverge:

• Citation units differ. Anthropic's API cites text spans. Perplexity cites whole sources by index. Adding them up mixes units.
• Retrieved is not cited. OpenAI's documentation says the number of sources a model consulted is often greater than the number it cites. A page can shape an answer without a citation.
• Surfaces differ. Google says AI Overviews and AI Mode may use different models and techniques. An API call to Gemini is neither of them.

The practical split is covered in prompt-based tracking vs engine-reported data. Use engine data for scale, tracker data for the content of answers.

Be plain with yourself and your team about this line.

In your control:

• Whether AI crawlers may fetch your pages. robots.txt rules are per bot: OAI-SearchBot governs ChatGPT search, GPTBot governs training only, and OpenAI says each setting is independent.
• Whether your firewall or CDN lets those crawlers through. A robots.txt allow means nothing if the edge returns a 403.
• Whether key facts are in the HTML before JavaScript runs. Most AI crawlers other than Googlebot do not execute JavaScript.
• Whether snippet controls such as noindex, nosnippet or data-nosnippet are hiding content by accident.
• Whether your own site states prices, plans and facts consistently.
• How you run the tracker: prompts, account state, cadence.

Not in your control:

• Whether an engine searches for a given query.
• How it rewrites the query and which pages it retrieves.
• Which retrieved pages it cites.
• What the model learned in training.

A tracker can show you outcomes in the second list. Your work happens in the first.

1. Check access before you measure. Confirm each AI search crawler is allowed in robots.txt and actually gets a 200 from your server. The AI Crawler Checker requests a page as each crawler and shows which ones your firewall turns away. Tracking a site that crawlers cannot reach measures nothing useful.
2. Pull engine-reported data. Read AI Overview impressions in Search Console (here's how to find them) and the AI Performance report in Bing Webmaster Tools. Export Bing's grounding queries if you have them.
3. Write a stable prompt set. Base it on real buyer questions and, where you have them, Bing grounding queries. Freeze it. Changing prompts resets your baseline.
4. Fix the measurement conditions. Run logged out, with memory off, from a stated location. Label every result with the engine, date and whether citations appeared.
5. Read each engine separately. Otterly saw one engine cite its test site thousands of times while four others barely cited it. A blended score would have hidden that. See whether one dashboard can track every engine.
6. Log wrong facts. Note every wrong price, outdated product name or competitor mix-up, with the answer that contained it. A correction tracker turns these into fixes on your own pages.
7. Re-run on a fixed cadence. Compare like with like over time, not one run against another tool's run. The free checker on this site gives a snapshot for up to three buyer questions, which is a starting point, not continuous tracking.

• Reading a position as a rank. No engine publishes one. Report presence, citation and accuracy instead.
• Averaging engines into one score. Different engines use different indexes and retrieval rules. Average them and you lose the signal.
• Assuming no citation means no mention in reality. The answer may have been ungrounded, or the page retrieved but uncited.
• Comparing your logged-in ChatGPT to the tracker. Memory and history change the rewritten query. You are comparing two different experiments.
• Treating a tracker's "Google AI" as AI Overviews. Many tools call the Gemini API, which is a different surface.
• Chasing tactics the evidence doesn't support. Ahrefs' matched test found AI Overview citations fell 4.6% after pages added schema. Google says it ignores llms.txt for Search. A dip in your tracker is not a cue to add markup.
• Changing the prompt set mid-trend. The line moves because the instrument changed, not because the engines did.

Is an AI SEO tracker worth using if it can't see real queries? Yes, if you use it for what it can do. A tracker with a fixed prompt set shows whether engines mention you, which sources they cite, and which facts they get wrong. That is enough to find errors and spot change over time. It is not enough to estimate traffic, reach or share of real demand. Pair it with Search Console and Bing Webmaster Tools for that side of the picture.

Can a tracker tell whether ChatGPT searched the web for an answer? Only indirectly. If the answer shows citations, live search ran. If it shows none, the answer may have come from training data, or search may have run without a visible citation. Trackers that call the API with search switched off are measuring training data only. Ask your tool how it calls each engine and record citation presence on every answer you analyse.

Why does my tracker show different results from what I see in ChatGPT? Your own session is logged in. OpenAI says ChatGPT may use saved memories when rewriting a search query, and product results consider memory and custom instructions. The tracker runs a clean session, may use a different model, and may or may not have search on. Both results can be accurate for their own conditions. Compare logged-out, memory-off runs only.

Does Google Search Console show AI Overview clicks? As of October 2026, no. Google's generative AI report in Search Console shows impressions for AI Overviews and AI Mode, broken down by page, country, date and device. It does not show clicks, click-through rate or position. Google has described more metrics as a future intention only. Impressions still tell you which pages appear, which is more than any prompt tracker can confirm.

How often should I re-run AI visibility checks? Often enough to catch change, rarely enough that each run uses the same conditions. Answers vary between runs, so one run is never a trend. A steady cadence with a frozen prompt set, fixed account state and fixed location gives you comparable points. Re-run sooner after you ship a fix, such as unblocking a crawler or correcting a price, so you can see whether answers change.