Logging Retrieved vs Cited Sources via API

Every AI answer built on live search has two source sets. The retrieved set is what the model pulled into its context. The cited set is what it chose to show the reader. Most tracking tools count only the second. The APIs give you both.

As of October 2026, here is what each one exposes:

OpenAI's own documentation says the number of sources is often greater than the number of citations. That gap is the point of logging both.

The two lists answer different questions. Being retrieved means your page reached the candidate pool for that query. Being cited means the model also surfaced it to the reader. A page can pass the first gate and fail the second. For the full distinction, see what retrieved sources are and cited vs retrieved: what AI citations really mean.

This works for any API that returns both lists. Each step makes the log comparable between runs.

1. Write a fixed prompt set. Use the buyer questions you actually care about. Keep the wording identical between runs.
2. Pin and record the model ID. Use an explicit model name, not a floating alias, and store it with every row. If you use an alias, log what it resolved to.
3. Turn web search on explicitly. On Perplexity's Agent API, web search is opt-in. Code still pointed at the retired Sonar Chat Completions endpoint (retired 2026-09-27 per Perplexity's announcement) can silently return ungrounded answers. Read Perplexity prompt tracking with the Agent API before you trust old scripts.
4. Store the raw response first. Save the full JSON untouched. You'll want to re-parse it when a field changes.
5. Extract the retrieved URLs. Normalize them: lowercase the host, strip tracking parameters, and keep the final URL after redirects.
6. Extract the cited URLs. On Perplexity, map each integer citation back to its entry in search_results. On Claude, map each cited span to its result URL and keep the cited_text. See how Claude picks sources for the span format.
7. Label every URL. Mark each one as retrieved and cited, or retrieved only. Flag your own domain, competitors and third-party pages that mention you.
8. Log the search queries if the response exposes them. OpenAI rewrites prompts into targeted queries, and Gemini generates its own. Those queries are often not your prompt.
9. Run each prompt more than once. Answers vary between runs. Record how many runs you did, so a single odd answer does not read as a trend.

Budget before you scale. Anthropic bills searches separately from tokens, and Gemini 3 bills per search query rather than per prompt.

If a row is missing one of these, you can't compare it to next month's row.

• Timestamp in UTC
• Platform and API endpoint
• Exact model ID returned in the response
• Retrieval state: search enabled, and whether it actually fired
• Prompt text and prompt ID
• Search queries the model ran, where exposed
• Full retrieved URL list, normalized
• Cited URL list, normalized
• Citation unit: span, URL or source index
• Cited text for span-level citations
• Date fields per result (page_age, date, last_updated)
• Answer text
• Run number for repeated prompts
• Raw response reference, so you can re-parse later
• Your own label per URL: own site, competitor, third party

Keep date fields as observations. They are publisher-declared, and no vendor says they drive source selection.

A citation is a display decision. It is not a full record of what shaped the answer.

Three reasons to treat counts as a noisy proxy:

• The retrieved set is larger than the cited set. Pages the model read but did not cite can still shape the wording. You only see them if you log retrieval.
• Citation is mechanically separable from use. Researchers studying attribution in RAG found that amplifying specific attention heads repaired more than 90% of missed citations without changing the sources. The model had the source either way. Only the citation changed.
• Units do not match across platforms. Claude cites text spans. Perplexity cites whole sources. Adding them into one number mixes different things.

The undercount is probably not random. It may favour well-known sources over smaller ones, so a small brand's real footprint can be larger than its citation count suggests. Report retrieved and cited counts separately, per platform, and never as one blended figure.

Your API log is a controlled experiment. The apps people use are not controlled, and they behave differently.

As of a September 2026 snapshot:

• ChatGPT decides at the model's discretion whether to search. Your API call searches because you told it to.
• Claude.ai search is a user toggle with no published default.
• Perplexity's free tier uses an auto-router, and paid tiers offer a menu of models. No single API model is the app.

Logged-in ChatGPT can also use saved memories when rewriting a search query, according to OpenAI's help documentation. Two users asking the same question can trigger different searches.

So read API logs as a proxy. They show what a given model retrieves and cites under fixed conditions. They do not show what a specific user saw. If you are deciding how to configure tests, read whether to test with web search on or off.

Each pattern points to a different next move.

Good: retrieved and cited

Your pages show up in both lists for your core prompts. Keep the facts on those pages current and stated plainly.

Mixed: retrieved but not cited

You reached the candidate pool and lost at generation. Don't rush to rewrite for AI. Benchmarks such as C-SEO Bench found most citation-oriented rewrites ineffective or harmful. What has a positive signal is stating specific facts (a price, a number, a one-line definition) on a fetchable page, which gives the engine a sentence it can quote.

Bad: never retrieved, competitors are

Check access first. Confirm robots.txt allows the search bots (OAI-SearchBot, PerplexityBot, Claude-SearchBot), confirm your firewall or CDN does not return 403 to them, and confirm key content is in the HTML before JavaScript runs. The AI Crawler Checker requests a page as each AI crawler and shows which ones get turned away.

Bad: third-party pages retrieved, yours never

When engines lean on other sites, being named accurately on those sites is associated with getting into the answer. No outcome is guaranteed.

Not a finding: empty retrieved list

Search did not fire or was not enabled. Fix the request before drawing conclusions.

Urgent: a retrieved source carries a wrong fact

That source may explain an error in the answer. Trace it with how to find which source caused a wrong AI fact.

Pick a fixed cadence and hold it. Monthly works for most brands. Weekly makes sense during a launch or a pricing change.

Also re-run outside the schedule when:

• The model ID in the response changes. Start a new baseline. Don't trend across models.
• You ship an access fix, such as a robots.txt change, a firewall rule or server-side rendering.
• A vendor changes an endpoint or a default, as Perplexity did with Sonar.

Compare only like with like: same prompts, same model, same retrieval state, same number of runs.