GPTBot vs OAI-SearchBot: Which One to Allow, and Why They Are Independent

OpenAI runs more than one crawler, and each user agent has its own purpose. As of October 2026, OpenAI's bots page describes three that site owners meet most often.

The key point is the middle row. OAI-SearchBot is the crawler tied to ChatGPT search. GPTBot is tied to training. If you want the details of every OpenAI agent in one place, see OpenAI's crawler documentation, digested.

The practical move: decide on training and on search as two separate questions, then write one robots.txt group for each bot.

The independence of the two settings is not an inference. OpenAI states it directly: "Each setting is independent of the others." Its own example is a webmaster who allows OAI-SearchBot to appear in search results while disallowing GPTBot so crawled content is not used for training.

On GPTBot, the page says it is "used to crawl content that may be used in training our generative AI foundation models" and that disallowing it "indicates a site's content should not be used in training." Nothing on the page claims that allowing GPTBot improves how you appear in ChatGPT search.

On OAI-SearchBot, the page says sites that opt out "will not be shown in ChatGPT search answers, though can still appear as navigational links." A navigational link is a plain link to a site, not a cited source inside a search answer.

The practical move: treat OpenAI's bots page as the reference for what each token does, and ignore advice that blends the two.

This is the most common version of the mistake. It sounds reasonable: ChatGPT is built on OpenAI's models, so feeding the training crawler should help. The evidence does not support it.

GPTBot governs training only. Training affects what a model knows with no web access, and that knowledge is frozen at a cutoff. OpenAI's model docs list a knowledge cutoff of 16 February 2026 for the GPT-5.6 family and 31 August 2025 for the Instant path, so anything crawled for training today reaches answers only in a future model generation, if at all.

Nobody has published a test of whether allowing or blocking GPTBot changes how often a model mentions a brand from memory. The honest label is unmeasured, in either direction. That means allowing GPTBot is a policy choice about training, not a visibility tactic.

The practical move: if your goal is to appear in ChatGPT search answers, check OAI-SearchBot first. Decide on GPTBot based on how you feel about training use.

The opposite myth stops many teams from blocking training at all. They fear that a Disallow for GPTBot will also hide them from ChatGPT search. OpenAI's statement that each setting is independent contradicts this.

If you want to keep your content out of training while staying eligible for search, write separate groups:

User-agent: GPTBot
Disallow: /

User-agent: OAI-SearchBot
Allow: /

The real risk sits elsewhere. A blanket rule that blocks every AI crawler, or a User-agent: * group with Disallow: / and no OAI-SearchBot group, removes search eligibility along with training. Under the Robots Exclusion Protocol (RFC 9309), a crawler follows the group that names it, and falls back to the * group only when none does.

The practical move: read your robots.txt for the exact tokens. Our walkthrough on blocking AI training but staying in AI search covers the same pattern for other vendors, and the per-bot robots.txt guide lists their tokens.

Being allowed is the first gate, not the outcome. It helps to separate the stages.

In your control

• Per-bot robots.txt permission. You decide whether GPTBot and OAI-SearchBot may crawl.
• Network reachability. OpenAI's help article on ChatGPT search says eligibility needs two things: allow OAI-SearchBot, and confirm your host or CDN allows traffic from OpenAI's published searchbot IP addresses. The word is "and". A firewall that returns 403 to OAI-SearchBot blocks you even when robots.txt allows it.
• Rendering. Vercel's crawler study found that ChatGPT and Claude crawlers fetch JavaScript files but don't execute them. Facts that appear only after JavaScript runs may be invisible to OAI-SearchBot.
• Snippet and index controls. Directives like noindex and nosnippet keep content out of answers on engines that document them.

Not in your control

• Whether ChatGPT searches at all. ChatGPT search is used at the model's discretion for each query.
• The query it runs. OpenAI says ChatGPT search typically rewrites your query into one or more targeted queries.
• Ranking. OpenAI's disclosure is that it ranks results using multiple factors, and placement is not guaranteed.

The practical move: fix every access gate you own, then stop expecting a crawler setting to produce a citation by itself.

Work through the gates in the order a request meets them.

1. Decide your training policy. Choose whether GPTBot may crawl. This choice does not affect ChatGPT search eligibility.
2. Confirm OAI-SearchBot is allowed. Open /robots.txt and find the group that applies to OAI-SearchBot. If it has none, check what the * group says. The free robots.txt check for ChatGPT search does this in one step.
3. Test the network layer. Request a real content page as OAI-SearchBot and record the status code. A 403, a 429, or a challenge page means your firewall or CDN is turning it away. Since 15 September 2026, Cloudflare's new defaults block Training and Agent crawlers on ad-displaying pages for some accounts while leaving Search allowed, so check even if you changed nothing. The AI Crawler Checker requests a page as each AI crawler and shows which ones get blocked, and our firewall testing playbook explains how to read the results.
4. Validate bots properly. OpenAI publishes OAI-SearchBot's IP ranges at openai.com/searchbot.json, but it also warns against relying on IP lists alone. Pair the user agent with verified-bot rules where your CDN offers them.
5. Check rendering. Compare your page's raw HTML with the rendered page. Move prices, product names and key facts into the server-rendered HTML.
6. Watch your logs. Confirm OAI-SearchBot actually visits after the changes.

Once access is clean, the ChatGPT search technical checklist covers the remaining items.

• Blocking GPTBot and assuming you're out of ChatGPT. You're out of training only. Search is governed by OAI-SearchBot.
• Allowing GPTBot to "get found". OpenAI asserts no search benefit from GPTBot anywhere on its bots page.
• One blanket AI block. A single rule that lists every AI user agent often removes OAI-SearchBot along with the training bots. This is the most common real-world error.
• Trusting robots.txt alone. Robots.txt can say allow while the firewall says 403. Nothing in a dashboard warns you.
• Expecting robots.txt to stop ChatGPT-User. OpenAI says robots.txt rules may not apply to user-initiated fetches. Use other controls if you need to restrict them.
• Reading a GPTBot log entry as search activity. If your logs show GPTBot but not OAI-SearchBot, that is a different situation; see why logs can show GPTBot but not OAI-SearchBot.

Once the gates are open, the free checker on this site shows whether ChatGPT names you for up to three buyer questions. Treat it as a snapshot, not a guarantee of future answers.