AI Visibility Tracking ToolFree checker

Does OAI-SearchBot need to crawl my site for ChatGPT to cite it?

99Visibility Research · Updated October 21, 2026 · 4 min read

Yes, for ChatGPT search answers. OpenAI's crawler documentation says sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though they can still appear as navigational links. Allowing the bot only makes you eligible. Your firewall or CDN must also let OpenAI's traffic through. Even then, ChatGPT decides per query whether to search and which pages to cite, and nobody can promise selection.

The short answer and why

For ChatGPT search, OAI-SearchBot access is a precondition. OpenAI's bot documentation says OAI-SearchBot "is used to surface websites in search results in ChatGPT's search features," and that "sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though can still appear as navigational links."

That sentence works in one direction only. Blocking the bot removes you. Allowing it does not add you. It puts your pages in the pool ChatGPT can draw from, and that's all.

If you want the bot itself explained, see what OAI-SearchBot is and how it works. The navigational-link exception is covered in whether ChatGPT can still link to a site that blocks OAI-SearchBot.

The two halves of the gate

OpenAI's help article on ChatGPT search frames eligibility as a conjunction. To make a site eligible, you "allow OAI-Searchbot to crawl the site and confirm that the website host or content delivery network allows traffic from OpenAI's published searchbot IP addresses."

Half one: robots.txt

robots.txt rules are per bot. A minimal setup that keeps search access while declining training looks like this:

User-agent: OAI-SearchBot
Allow: /

User-agent: GPTBot
Disallow: /

The common mistake is a blanket AI block copied from a template. It silently removes ChatGPT search eligibility along with training. Our guide to GPTBot vs OAI-SearchBot walks through the difference.

Half two: the network

A firewall, CDN or bot-management rule can return a 403 or a challenge page to a crawler that robots.txt allows. Nothing in your analytics flags it. OpenAI publishes the OAI-SearchBot IP ranges as a JSON file, but it also warns against relying on IP allowlisting alone, so verify on user agent plus your CDN's verified-bot settings. If you use Cloudflare, check your AI crawler categories: new defaults took effect on 15 September 2026 for some accounts, with Search crawlers left allowed by default and Training and Agent crawlers blocked on ad-displaying pages.

The step-by-step test is in testing whether your firewall lets AI crawlers in.

What eligibility does not buy

Passing both halves gets you considered. Three things stay outside your control.

  • Whether ChatGPT searches at all. As of October 2026, ChatGPT search is model-discretionary. Many answers come from training data with no web lookup, and OAI-SearchBot has no effect on those.
  • Which queries it runs. OpenAI says ChatGPT search typically rewrites your prompt into one or more targeted queries. You can't see them.
  • How it ranks results. OpenAI's full disclosure is that ChatGPT ranks results using "multiple factors" and that placement is not guaranteed.

An independent test shows the gap. In OtterlyAI's 2026 experiment, two sites ranked in Bing and ChatGPT's crawlers kept visiting them, yet ChatGPT cited them effectively zero times. Crawling and indexing were necessary. They weren't enough.

Edge cases worth knowing

  • ChatGPT-User is a different bot. It fetches pages when a user asks ChatGPT to. OpenAI says it is not used to decide what appears in search, and that robots.txt rules may not apply to it.
  • Blocking GPTBot does not hide you from search. OpenAI states each setting is independent: you can allow OAI-SearchBot and disallow GPTBot.
  • JavaScript-only content. Vercel's crawler study found ChatGPT's crawlers fetch JavaScript files but don't execute them. If your prices or product facts only appear after hydration, OAI-SearchBot can reach the page and still miss them.
  • No submission form exists. There's no way to request inclusion beyond being reachable.

What to do next

  1. Read your live robots.txt and confirm OAI-SearchBot is allowed on the pages that matter. The free ChatGPT search block check does this in one step.
  2. Request a key page as OAI-SearchBot and confirm it gets a 200, not a 403 or a challenge. The AI Crawler Checker shows which crawlers your firewall turns away.
  3. View the raw HTML with JavaScript off and confirm your load-bearing facts are in it.
  4. If all three pass and you still don't appear, work through what else can block ChatGPT when robots.txt allows AI bots.

These steps remove blocks. They can't guarantee a citation, and anyone who promises one is guessing.

More on this

If I allow OAI-SearchBot, will ChatGPT cite me? Not necessarily. Allowing it makes your pages eligible for ChatGPT search. ChatGPT still decides per query whether to search, rewrites the query itself, and ranks results on factors OpenAI describes only as "multiple factors" with no guaranteed placement. OtterlyAI's 2026 experiment found sites that ChatGPT's crawlers visited and Bing ranked still received effectively zero ChatGPT citations.

Does allowing GPTBot help ChatGPT find my site? No. OpenAI's documentation describes GPTBot as a training crawler and asserts no search benefit from allowing it. OAI-SearchBot governs ChatGPT search. OpenAI says each setting is independent, so you can block GPTBot to decline training use and still allow OAI-SearchBot for search eligibility.

Can ChatGPT mention my brand if I block OAI-SearchBot? Yes, in two ways. OpenAI says opted-out sites can still appear as navigational links. Separately, answers ChatGPT gives without searching draw on what the model learned in training, frozen at a knowledge cutoff. Blocking OAI-SearchBot keeps your pages out of search answers but does not erase what the model already knows.

How do I know if my firewall blocks OAI-SearchBot? Request a real content page using OAI-SearchBot's documented user-agent string and record the response. A 200 with your content is a pass. A 403, a 429, or an HTML page containing a CAPTCHA or JavaScript challenge is a fail. Your server or CDN logs, filtered to verified bots, show the same thing from the inside.

Does robots.txt control ChatGPT-User too? Not reliably. OpenAI says ChatGPT-User acts on behalf of a user and that robots.txt rules may not apply to it. It is also not used to decide whether content appears in ChatGPT search. Treat OAI-SearchBot as the control for search eligibility and ChatGPT-User as a separate, user-triggered fetcher.

Sources

  1. Overview of OpenAI Crawlers
  2. Searching the web with ChatGPT
  3. OpenAI
  4. The rise of the AI crawler - Vercel
  5. 2000 AI Blogs Experiment: Does Google Penalize AI Content?

This page is AI-assisted. It was written from a verified evidence base and checked against our editorial policy. Engine behaviour changes often; facts are as of October 21, 2026.

Keep reading