GPTBot: definition and what it controls

GPTBot is the user agent OpenAI uses to crawl public web content that may go into training its generative AI foundation models. OpenAI's bots page describes it as the crawler used "to make our generative AI foundation models more useful and safe."

The same page gives the consequence of blocking it: disallowing GPTBot "indicates a site's content should not be used in training generative AI foundation models." That is the whole scope. GPTBot is a training crawler, not a search crawler. For the full list of OpenAI's bots, see our digest of OpenAI's crawler documentation.

GPTBot identifies itself with the token GPTBot in its user agent string. You control it in robots.txt like any other named crawler. A site that wants to stay out of training but stay eligible for ChatGPT search would write:

User-agent: GPTBot
Disallow: /

User-agent: OAI-SearchBot
Allow: /

Three mechanics matter in practice:

• Rules are per bot. A rule for GPTBot says nothing about OAI-SearchBot or ChatGPT-User. Write each one explicitly.
• It reads raw HTML. Vercel's crawler study found that ChatGPT's crawlers fetch JavaScript files but don't execute them, so content that only appears after JavaScript runs is invisible to them.
• robots.txt is not the only gate. A firewall or CDN can turn away a crawler that robots.txt allows. Before trusting a log line that claims to be GPTBot, verify the hit is real.

GPTBot sits in the parametric stage: what a model knows with no web access. Any effect of being crawled for training only arrives with a future model generation. As of October 2026, OpenAI documents a February 16, 2026 knowledge cutoff for its GPT-5.6 models, so recent pages are not in that knowledge regardless of your GPTBot setting.

Nothing published measures whether allowing or blocking GPTBot changes how often a model recalls your brand. OpenAI does not claim that allowing it helps retrieval or citation. The honest label is unmeasured, not helpful and not harmful.

The crawler that decides ChatGPT search eligibility is OAI-SearchBot. OpenAI says sites opted out of it "will not be shown in ChatGPT search answers, though can still appear as navigational links." OpenAI's help center adds that your host or CDN must also allow traffic from its published searchbot IP addresses. So the next move is to set your GPTBot rule on licensing grounds, then confirm OAI-SearchBot is allowed in robots.txt and actually reachable. The AI Crawler Checker requests a page as each AI crawler and shows which ones your firewall turns away.

• "Allow GPTBot so ChatGPT can find you." GPTBot governs training only. Search uses OAI-SearchBot.
• "Blocking GPTBot removes you from ChatGPT." OpenAI states the settings are independent: a webmaster can allow OAI-SearchBot to appear in search results while disallowing GPTBot.
• "A blanket AI block is the safe default." A rule that blocks every OpenAI agent removes ChatGPT search eligibility along with training. It's one of the most common real-world errors. Separate training bots from retrieval bots.
• "robots.txt stops every OpenAI fetch." OpenAI says robots.txt rules may not apply to ChatGPT-User, because a user starts those requests.
• "Token spelling doesn't matter." Match OpenAI's documented token exactly. See whether bot names in robots.txt are case-sensitive.

• ChatGPT-User: OpenAI's fetcher for actions a user starts in ChatGPT.
• OAI-SearchBot: OpenAI's crawler for ChatGPT search inclusion.
• ClaudeBot: Anthropic's training crawler, the closest equivalent to GPTBot.
• Meta's crawlers: Meta-ExternalAgent and Meta-WebIndexer split training and search the same way.
• Google-Extended: Google's token for Gemini training and grounding, which does not affect Google Search inclusion.

What is the GPTBot user agent token? The token to target in robots.txt is GPTBot. OpenAI lists it on its crawler documentation page alongside OAI-SearchBot and ChatGPT-User, each with its own purpose. Use the exact token OpenAI documents and write a separate User-agent group for each bot you want to control, rather than relying on a wildcard rule to cover them all.

Should I block GPTBot? It's a licensing decision, not a visibility one. Block GPTBot if you don't want your content used to train OpenAI's foundation models. Allow it if you're comfortable with that. Either way, the effect on how models describe your brand is unmeasured by anyone, including OpenAI. What matters for ChatGPT search is your OAI-SearchBot rule and whether your firewall lets it through.

Does blocking GPTBot stop my content appearing in ChatGPT answers? Not in ChatGPT search. OpenAI states each crawler setting is independent, so a site can block GPTBot and still appear in search results through OAI-SearchBot. Blocking GPTBot only signals that crawled content should not be used in training. Content already learned by released models stays in those models until they are replaced.

Can GPTBot read content rendered by JavaScript? Vercel's crawler study found ChatGPT's crawlers download JavaScript files but don't execute them, so they can't read client-side rendered content. If your key facts, such as pricing or product names, only appear after JavaScript runs, put them in the server-rendered HTML. The same applies to OAI-SearchBot, which matters more for search answers.