How Claude Picks Sources: The Search Tool and Its Citations

Claude doesn't search the web for every question. Retrieval is per query and discretionary, and as of October 2026 three different things govern it depending on where Claude runs.

• In the Claude API, Claude decides whether to call the web search tool. The integrator's system prompt steers that decision. The same question can trigger a search in one app and not in another.
• On claude.ai, web search is a user toggle. Anthropic publishes no default for it, so you can't assume a given consumer answer was grounded.
• Org-level API defaults don't describe consumer behavior. A setting that applies to developers says nothing about what a claude.ai user sees.

This matters because a Claude answer comes from one of two places. With search off, Claude answers from what the model learned in training, frozen at a cutoff. Nothing you publish this quarter reaches that answer. With search on, your page can enter the answer through retrieval and citation. Our guide to how AI search picks sources separates those stages in more detail.

The practical rule: if a Claude answer shows no citations, you don't know whether it searched. Treat uncited answers as possibly parametric, and don't read them as a verdict on your site.

Anthropic's web search tool documentation lists the fields each result returns. As of October 2026, those include:

• url: the address of the page
• title: the page title
• page_age: described by Anthropic as "When the site was last updated"

The page_age field gets a lot of attention, and it's easy to over-read. Exposure is not weighting. Anthropic does not state that page_age affects which results Claude uses or cites. The date is also usually something the publisher declares, which makes it easy to change without changing the page.

The freshness studies that circulate don't close the gap. Seer's July 2026 study found 75% of cited pages had been updated within a year, but it had no control group of uncited pages. Without a base rate for the general web, that number can't show that engines reward freshness.

What to do: keep your visible date, sitemap lastmod and Last-Modified header consistent with each other. Treat that as hygiene. Don't bump dates hoping Claude will pick the page. For a full field-by-field read, see what Claude's web search tool docs reveal.

Claude cites text spans, not whole sources. Each citation carries a cited_text excerpt of up to 150 characters from the page it used. Perplexity, by contrast, cites whole sources by an integer index. The two units don't measure the same thing, which is why a single blended score across engines can mislead. Our explainer on whether AI engines cite whole pages or sentences compares the formats.

Three consequences follow for a marketing lead.

The quotable sentence is the unit

Because Claude quotes a short span, a clear factual sentence on a fetchable page gives it something to quote. Stating a specific fact (a price, a number, a one-line definition) is the only on-page practice with a positive experimental signal. That is a reason to state facts plainly, not a promise of a citation.

Retrieved is not the same as cited

The API exposes the results Claude retrieved separately from the spans it cited. A page can be retrieved and never cited. Mechanistic research (arXiv:2606.28358) also found citation output can be changed without changing the sources, so a citation count is a noisy proxy for influence. If you run queries through the API, log retrieved and cited sources separately.

Spans let you check accuracy

A cited_text excerpt shows exactly which sentence Claude leaned on. Compare it with your page. If Claude states a wrong price next to a span from an outdated page, you know which page to fix.

Simon Willison reported in March 2025 that Anthropic's Trust Center listed Brave Search as a subprocessor, around the launch of Claude web search. That's a meaningful signal about where Claude's results may come from. It is not a current, documented description of how Claude retrieves pages, and it doesn't tell you how results are ranked or filtered before Claude sees them.

Read it this way:

• Being findable in the index Claude queries is a precondition for being retrieved.
• Anthropic publishes no list of ranking factors for its search results.
• The query Claude sends may not match what the user typed. Engines commonly generate their own search queries, and nobody outside the operator sees them.

If you want to dig into this, start with what search engine Claude uses. Then check whether your key pages appear for the topics you care about in that index, without assuming that a result there guarantees a Claude citation.

The levers with evidence behind them are access levers. They work by removing blocks, not by adding markup. Anthropic documents separate agents with separate roles:

• ClaudeBot: collects content for training.
• Claude-SearchBot: crawls to improve Claude's search results.
• Claude-User: fetches pages when a user's request triggers it.

robots.txt rules are per agent. A blanket AI block written to stop training can also shut out search crawling. If you want to opt out of training but stay eligible for Claude search, a rule set looks like this:

User-agent: ClaudeBot
Disallow: /

User-agent: Claude-SearchBot
Allow: /

Two further gates sit below robots.txt.

1. JavaScript rendering. Vercel found Claude's crawlers fetch JavaScript files (23.84% of requests) but don't execute them, and an independent retest in June 2026 found Claude-SearchBot doesn't render JavaScript either. Pricing or product facts that only appear after hydration may be invisible to Claude. Put them in the server-rendered HTML.
2. Firewall and CDN. A WAF or bot manager can return 403 to a crawler that robots.txt allows. Anthropic warns that IP blocking "impedes our ability to read your robots.txt file." The AI Crawler Checker requests a page as each AI crawler and shows which ones your firewall turns away.

One caution: third-party pages can carry injected instructions aimed at AI agents. That's a manipulation tactic and a reputational risk if it targets your brand. Never use it on your own site.

Be clear about the limits before you act on any Claude visibility number.

The consumer app also moves. A 99Visibility snapshot from September 2026 recorded Sonnet 5 as the reported default for Free and Pro (a secondary source) and Opus 5 as selectable. The model a user gets can change without notice.

So a low Claude score often reflects measurement conditions as much as your site: search off, a different model, or a query Claude never searched for. Our note on why Claude scores low in a visibility report walks through the causes. Then fix what you control: crawler access, server-rendered facts, and consistent facts across your own pages. You can check whether AI quotes your site accurately using the cited spans.