Prompt-Based Tracking vs Engine-Reported AI Data

The two methods answer different questions, so their numbers rarely agree.

Prompt-based tracking sends a fixed set of prompts to each AI engine on a schedule, then records whether your brand is mentioned or your URL is cited. You choose the prompts. The count is a sample: it can only see answers to questions you thought to ask.

Engine-reported data is what an engine tells site owners about itself. Two first-party reports matter today:

• Bing Webmaster Tools AI Performance (public preview since 10 February 2026, expanded in June 2026) reports total citations, cited pages, grounding queries, intents, topics and Citation Share across Microsoft Copilot, Bing AI summaries and select partner integrations.
• Google Search Console's generative AI report shows impressions for AI Overviews and AI Mode, broken down by page, country, date and device. It has no clicks, no CTR and no position.

The practical difference: a tracker samples the outside of the engine. A report counts from the inside, but only for that engine. For a fuller picture of tracking across models, see LLM visibility tracking.

OtterlyAI ran a public experiment in 2026 with two new domains, each loaded with roughly 1,000 AI-generated posts. It tracked AI citations across seven engines with its own prompt-based tracker and also pulled Bing Webmaster Tools data.

For one site, capybara-world.com, Otterly's tracker logged 3,934 citations, of which 2,718 (69%) came from Microsoft Copilot. Microsoft's AI Performance report credited the same site with 42,817 citations, more than ten times the tracker's count.

Read that gap carefully:

• The time windows may not match. Otterly did not publish the two counts over an identical period, so this is not an exact ratio.
• It still shows what a prompt sample is. A tracker counts answers to the prompts it asks. Copilot answered many questions Otterly never asked.
• It covers one engine. The 42,817 figure is Microsoft's view only. No comparable report exists for ChatGPT, Perplexity or Claude in the evidence we hold.
• A vendor ran it. Otterly sells AI visibility tracking, and so does 99Visibility. That doesn't make the data wrong, but it is why replication matters.

If your own numbers diverge like this, why Bing's AI citation counts differ from tracking tools walks through the usual causes.

The deeper problem is not sample size. It is that the prompt you send is not the query the engine runs.

• ChatGPT rewrites. OpenAI's help center says ChatGPT search typically rewrites your query into one or more targeted queries that it sends to search providers. If memory is on, saved memories can shape that rewrite.
• Google fans out. Google says AI Overviews and AI Mode may use a query fan-out technique, issuing multiple related searches across subtopics.
• Gemini generates its own. Google's Gemini API documentation says grounding automatically generates one or more search queries and executes them.

So a prompt tracker measures how an engine responds to your wording, after a hidden rewriting step you cannot see. Two trackers with different prompt sets can report different visibility for the same brand, and both are internally correct.

Retrieval also fires at the engine's discretion. ChatGPT decides per query whether to search, and some answers come from the model's training data with no live lookup at all. Any tracker number averages over that mixture.

What this means for you: treat a prompt set as a deliberately chosen probe, not a census. How to build a prompt set covers choosing prompts that reflect real buyer questions, and whether prompt-tracking tools are accurate covers the limits in more depth.

Neither column is the truth. Each sees a different slice.

Engine reports are closer to reality, but they carry their own caveats.

• Sampling on Bing. Microsoft says the grounding queries shown are a sample of overall citation activity. That disclosure sits with the grounding-query metric specifically, so don't assume every number in the report is sampled.
• Citation Share is not a scoreboard. Microsoft describes it as an observational metric, not a ranking system. Its denominator is dominated by whoever already wins the query, so it can fall while your absolute citations rise.
• No API for the AI data. As of the documentation reviewed in August 2026, Bing's API exposed none of the AI Performance metrics, and Google's Search Analytics API did not cover the generative AI report. Microsoft has signalled that API access may come. Until then, you export by hand.
• Google's report can't show outcomes. Impressions with no clicks or CTR cannot tell you whether AI visibility produced traffic. A Google-side bug also depressed impressions from around 12 August 2026, so trends from that window need a check.
• Citation is not use. OpenAI notes that the number of sources a model consults is often greater than the number it cites. Counts in either method undercount influence.

What this means: use engine reports to learn what real queries look like, not as a verdict on how well you're doing.

Use both, for different jobs.

Use engine-reported data when

• You want to know the queries Copilot and Bing actually used to retrieve your pages. Export Bing's grounding queries and compare them with the prompts you track.
• You need a volume figure for one engine. Bing's count will usually dwarf a tracker's sample.
• You want to know whether Google shows your pages in AI Overviews or AI Mode at all. Search Console impressions answer that, by page.

Use prompt-based tracking when

• You need coverage of ChatGPT, Perplexity, Claude or Gemini, which publish no comparable site-owner report in our evidence.
• You want to see which competitors appear beside you for buyer questions.
• You want to check whether answers state your prices, products or facts correctly.
• You need a repeatable baseline. Run prompts logged out with memory off, as described in ChatGPT prompt tracking.

Combine them

1. Pull grounding queries from Bing Webmaster Tools.
2. Rewrite your prompt set so it overlaps with that real language.
3. Track each engine separately rather than as one blended score. One dashboard across all engines explains why blending hides differences.
4. Report presence and citation counts, never a rank.

If you want a starting snapshot before committing to either, the free checker on this site runs up to three buyer questions and scores the answers. It is a one-time look, not continuous tracking.

Is the 42,817 vs 3,934 gap a reliable ratio? No. It shows the scale difference between a prompt sample and an engine's own count, but Otterly's two figures may cover different time windows. It also comes from one site in a low-competition topic. Treat it as evidence that prompt samples miss most activity, not as a multiplier you can apply to your own tracker numbers.

Can I see the real queries ChatGPT runs for my brand? Not as a site owner. OpenAI documents that ChatGPT search rewrites prompts into targeted queries, but it does not publish those queries to publishers. Bing Webmaster Tools is the only first-party source in our evidence that exposes grounding queries, and it covers Copilot, Bing AI summaries and select partners, not ChatGPT itself.

Does a higher prompt-tracking score mean I'm doing better in AI search? Not necessarily. A score reflects answers to the prompts you chose, after hidden query rewriting, across a mix of answers with and without live retrieval. It can move because your prompt set changed or the engine changed its routing. Compare like with like over time, keep the prompt set stable, and read each engine separately.

Does Google report my position in AI Overviews? No. Google's generative AI report in Search Console shows impressions only, with no clicks, CTR or position. Google also says third-party tools don't have access to its internal ranking data. Any tool reporting an AI Overviews rank has no ground truth to check that number against.