Web search tool: how LLM APIs search the web

A web search tool is a capability a developer attaches to an LLM API request. It lets the model issue search queries, read the results, and return an answer with citations to the pages it drew on. The term also covers grounding features such as Gemini's Grounding with Google Search.

The key word is tool. The model is offered the ability to search. It is not forced to. For a full answer on defaults by vendor, see whether AI model APIs search the web by default.

The flow is similar across vendors, as of October 2026:

1. The developer enables the tool in the request.
2. The model decides whether the question needs live information.
3. If it does, the model writes its own queries. OpenAI says ChatGPT search rewrites a prompt into one or more targeted queries, and Google says the Gemini API automatically generates one or more search queries.
4. Results come back into the model's context, and it writes an answer with citations.

Vendors differ in the details:

Perplexity's older Sonar Chat Completions endpoint retired on 27 September 2026. Code that moved to the Agent API without enabling web search returns ungrounded answers without any error. The Sonar to Agent API migration notes cover this, and the Claude web search tool documentation explains Anthropic's version.

Whether search runs decides which version of the model you are hearing from. With search off, the model answers from training data. OpenAI lists a 16 February 2026 knowledge cutoff for its GPT-5.6 models and 31 August 2025 for the Instant path used in ChatGPT. Nothing you publish now reaches those answers.

With search on, your pages can enter the answer, but only if a search could return them. That puts the access basics back in play: crawler permission, a firewall that lets the bots through, and key facts present in the HTML before JavaScript runs.

Two measurement points follow:

• Retrieved is not cited. OpenAI's documentation says the number of sources is often greater than the number of citations. Logging retrieved vs cited sources shows how to capture both.
• Any test should state its retrieval setting. Read whether visibility tools should test with search on or off before trusting a score.

The API answer is what users see

It isn't. ChatGPT searches at the model's discretion, and in Claude.ai web search is a user toggle with no published default. Gemini API grounding is also not Google AI Overviews or AI Mode. The API vs consumer app comparison sets out the gaps.

The prompt is the query

The model rewrites prompts into its own queries. Optimizing for the exact prompt you typed targets an input the engine may never run.

Search results are trustworthy input

Research on search agents (arXiv 2606.16821) found that manipulated content in results can steer answers, measured on a mock search API. Hidden or injected instructions are a manipulation tactic, not a strategy. See LLM search agents and manipulated web content.

• Grounding: tying a model's answer to retrieved documents rather than training data alone.
• Knowledge cutoff: the date after which a model's training data stops.
• Retrieval-on vs retrieval-off: whether an answer used live search.
• Citation: the source a model attributes, which is a noisy proxy for what it actually used.
• OAI-SearchBot, PerplexityBot, Claude-SearchBot: crawlers that build the indexes search features draw from.

Is the web search tool on by default in LLM APIs? Usually not in the way people assume. Perplexity's Agent API requires opting web search in. On OpenAI and Anthropic the developer attaches the tool, and the model then decides whether to search for a given request. Settings can change, so check each vendor's current documentation, as of October 2026, before relying on a default.

Does the web search tool cost extra? Often, yes. Anthropic charges $10 per 1,000 searches plus token costs, and searches that error are not billed. Google bills Gemini API grounding per search query on Gemini 3 models and per prompt on 2.5 and older. Because the model chooses how many queries to run, one request can trigger more than one billed search.

Can I see which pages the model read but didn't cite? On some APIs. OpenAI's Responses API returns a sources list of every URL consulted, separate from citations. Anthropic and Perplexity also return search result sets alongside citations. Comparing the two shows whether your page is being found but dropped, which is a different problem from not being found at all.

Why do two runs of the same question give different sources? Retrieval is per query and discretionary. The model may search in one run and not the next, may write different queries each time, and search indexes change. Personal context such as ChatGPT memory can also shape queries in consumer apps. Treat any single answer as a sample, and run repeated tests with the retrieval setting recorded.