noindex: definition and how it works for AI search

noindex is an instruction from the site owner to a search engine: you may crawl this page, but do not store it in your index or show it in results. It is one of several values of the robots meta tag, alongside nosnippet and max-snippet.

Google and Bing both document noindex as an on-page control. It works per page. It does not affect other pages on the same site.

There are two ways to deliver the directive.

In the HTML

Place a meta tag in the page head:

<meta name="robots" content="noindex">

In the HTTP response

Send a header with the page, which also works for PDFs and other non-HTML files:

X-Robots-Tag: noindex

The crawler fetches the page, reads the directive, and drops the page from its index. Two details matter:

• Where the tag appears. Most AI crawlers do not execute JavaScript, according to Vercel's crawler study. A noindex added by a script may be read by Googlebot, which renders pages, and missed by others. See whether AI crawlers see robots meta tags added by JavaScript.
• Where the header comes from. A CDN or server rule can add X-Robots-Tag to a whole path without anyone touching the page. Does an X-Robots-Tag header keep pages out of AI search covers that case.

Google's AI features documentation states that to be shown as a supporting link in AI Overviews or AI Mode, a page must be indexed and eligible to be shown in Google Search with a snippet. It adds that there are no additional technical requirements. A noindexed page fails the first condition, so it cannot be a supporting link on either surface.

That makes noindex one of the few levers with a documented, mechanical effect. It is also subtractive. Removing an unwanted noindex restores eligibility, not inclusion. Google says indexing and serving isn't guaranteed.

The real risk is accidental. Say you launch a redesign. The staging site carried noindex in the shared layout so it would stay out of search, and the flag shipped to production. Every page built on that layout is now out of Google, and out of its AI features, with no warning in the page itself. Templates spread one decision across hundreds of URLs.

The next move: check the raw HTML and the headers for each template type (home, product, pricing, blog, docs). The accidental noindex audit playbook walks through it step by step.

• "noindex and nosnippet are the same." They are not. nosnippet keeps the page indexed but stops its text from being shown in snippets and AI answers. noindex removes the page from the index entirely.
• "Blocking a crawler in robots.txt is the same as noindex." robots.txt controls whether a bot may fetch a page. noindex controls whether a fetched page is stored. They are different controls with different effects.
• "noindex stops AI training." Training is governed by separate tokens. Google-Extended controls whether content is used for Gemini training and grounding, and it does not affect Search inclusion. GPTBot governs OpenAI training only.
• "There is an AI-only noindex." As of October 2026, Google has no robots.txt token or meta value that removes a page from AI Overviews while keeping it in Search. Snippet controls and Search Console settings are the documented options.
• "Removing noindex gets you cited." It removes a block. What the engine then retrieves and cites is not in your control.

• Robots meta tag: the tag that carries noindex and other directives.
• nosnippet: keeps a page indexed but out of snippets and AI summaries.
• X-Robots-Tag: the HTTP header version of the same directives.
• data-nosnippet: an HTML attribute that hides one section from snippets and AI summaries on Google and Bing.
• Google-Extended: the Google token for Gemini training and grounding, separate from Search indexing.

How do I check whether a page has noindex? View the raw page source and search for a meta robots tag containing noindex. Then inspect the response headers for X-Robots-Tag, because a server or CDN can add it without changing the HTML. Check one URL per template, since templates share the same head. Google Search Console's URL Inspection also reports whether Google indexed the page and why not.

Does noindex remove a page from ChatGPT or Perplexity? noindex is a search index directive, and Google and Bing document how they honour it. OpenAI and Perplexity control their search eligibility mainly through their own crawler tokens in robots.txt, such as OAI-SearchBot and PerplexityBot. Do not assume noindex and those tokens behave identically. Check each engine's documented controls separately.

Should I use noindex or nosnippet to keep a page out of AI Overviews? Use nosnippet or max-snippet:0 if you want the page to stay in regular Search but keep its text out of AI answers. Use noindex only if you want the page out of Google Search entirely. noindex is the blunter tool, and it costs you classic search traffic as well as AI feature eligibility.

How long after removing noindex does a page return? The page returns only after the crawler revisits it, sees the directive is gone, and indexes it again. Google does not guarantee indexing or a timeline. Requesting indexing in Search Console can prompt a recrawl. Eligibility for AI Overviews follows indexing, but appearing in them is still not guaranteed.