How to Appear in AI Search Results: Get Eligible First

An AI answer that mentions your brand passes through four stages. Knowing which one you're dealing with tells you what you can fix.

1. Parametric. What the model knows with no web access, frozen at its training cutoff. Nothing you publish this quarter reaches it.
2. Access. Whether a crawler may fetch and store your page at all. You and your CDN control this.
3. Retrieval. Whether your page enters the model's context for a given query. The engine controls this, partly through its index.
4. Generation. Which retrieved pages the model uses and cites. The model controls this.

Retrieval does not fire every time. As of October 2026, Google AI Overviews often don't trigger, ChatGPT searches at the model's discretion, and web search in Claude.ai is a user toggle. ChatGPT also doesn't send the user's exact words to its search providers. OpenAI says ChatGPT search "rewrites your query into one or more targeted queries."

What this means: you can't optimize for a prompt you can't see. You can make sure that when an engine does search, your pages are fetchable, readable and in its index. That is the job of this guide.

Be clear-eyed about the split before you spend money.

In your control

• Which bots robots.txt allows, one by one.
• Whether your firewall, CDN or bot-management setting lets those bots through.
• Whether your important content exists in the raw HTML.
• Whether pages carry noindex, nosnippet or similar controls by accident.
• Whether Google and Bing have your pages indexed.
• Whether your own site states key facts (prices, product names, definitions) clearly and consistently.

Not in your control

• Whether an engine searches the web for a given question.
• How it rewrites the question and what it retrieves.
• Where your page lands in the model's context, and whether it gets cited.
• What the model learned at training time.

The C-SEO Bench study found that a document's position in the model's context mattered far more than rewriting the content, and no publisher can set that position. So treat the controllable list as hygiene. Each item removes a reason to be left out. None of them guarantees a mention. For the wider picture of how classic search fits in, see SEO for AI search.

1. Allow the retrieval bots in robots.txt

Each AI company runs separate bots for training, search indexing and user-triggered fetches. OpenAI states that "each setting is independent of the others." A blanket AI block is the most common way sites remove themselves from ChatGPT search without meaning to.

A site that wants search visibility but no training use might write:

User-agent: OAI-SearchBot
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Claude-SearchBot
Allow: /

User-agent: GPTBot
Disallow: /

User-triggered fetchers behave differently. OpenAI says robots.txt "may not apply" to ChatGPT-User, and Perplexity says Perplexity-User "generally ignores robots.txt rules."

2. Confirm your CDN and firewall let them through

robots.txt is a request, not a gate. OpenAI's help centre says eligibility means allowing OAI-SearchBot and confirming your host or CDN allows traffic from OpenAI's published searchbot IP addresses. Bot-management tools can return a 403 or a challenge page to a bot your robots.txt welcomes. That failure is silent.

Cloudflare changed its defaults on 15 September 2026: Training and Agent crawlers are blocked by default on ad-displaying pages for new domains, new sites on existing accounts and existing free-tier customers, while Search stays allowed. If you're on Cloudflare, check your AI crawler settings even if you changed nothing. To test, request a page with each crawler's user agent and record the status code. The AI Crawler Checker does this and shows which bots your firewall turns away.

3. Render key content in the HTML

Vercel's crawler study found that ChatGPT and Claude crawlers fetch JavaScript files but "don't execute them. They can't read client-side rendered content." Googlebot renders JavaScript, so this mostly affects non-Google engines. Fetch your pricing, product and about pages with JavaScript off. If prices, plan names or your one-line description are missing, move them into server-rendered HTML. Check tabs, accordions and "load more" sections too.

4. Stay indexed and snippet-eligible in Google

Google states that to appear as a supporting link in AI Overviews or AI Mode, a page "must be indexed and eligible to be shown in Google Search with a snippet." It adds that there are no additional technical requirements. There is no robots.txt token for AI Overviews or AI Mode: Googlebot rules are the control. Search Console also has a generative AI inclusion control. Inclusion is the default, but a child property can inherit an opt-out from its parent. Check it on every property you own. Platform detail is in getting cited in AI Overviews.

5. Stay indexed in Bing

Microsoft Copilot grounds its answers in the Bing index. In OtterlyAI's 2026 experiment, Google deindexed two AI-generated test sites, Bing kept them, and Copilot kept citing them while Google's AI surfaces almost never did. Indexes are separate. Verify your site in Bing Webmaster Tools and check index status there, not just in Search Console.

6. Remove accidental suppression

noindex, nosnippet, max-snippet:0 and data-nosnippet keep content out of AI answers. Bing says content marked data-nosnippet stays indexed but is excluded from snippets and AI summaries. These often arrive through a template, for example a tag meant for a cookie banner that wraps the whole page. Search your HTML and headers for each one and confirm every use is intentional.

Work from the gate that blocks the most to the one that blocks the least.

1. Network access first. A 403 at the CDN cancels everything else for that bot. Test each retrieval bot against a real content page and against /robots.txt itself.
2. robots.txt second. Split training bots from retrieval bots. Allow the retrieval bots for the engines you care about.
3. Suppression controls third. Remove stray noindex and snippet directives, and check the Search Console generative AI control.
4. Rendering fourth. Fix the pages that carry your key facts first: pricing, product, about, comparison pages.
5. Index status last. Confirm Google and Bing hold the pages you fixed, then recheck after the next crawl.

How long results take is not something any published study measures, so don't plan around a promised date. Fixes to access take effect only after crawlers return. For what to expect, read how long AI visibility work takes. Engine-specific checks are in getting cited by ChatGPT and the Perplexity checklist.

Eligibility gets you into the candidate pool. A few things have some evidence for what happens next, and none comes with a promised effect.

• State specific facts plainly. A price, a number or a one-line definition on a fetchable page gives the engine a sentence it can quote. This is the only positive experimental signal for on-page content.
• Be named where engines already look. When engines cite review sites, directories or comparison articles instead of you, being named on those pages is the route into the answer. The evidence shows an association, not a guaranteed effect. See getting recommended in your category.
• Stay on topic. Independent studies agree that topical relevance to the query is one of the most reproducible drivers of citation. That means covering the subject well, not stuffing keyword variants.
• Keep facts consistent. If your pricing page, footer and about page disagree, an engine can repeat the wrong one.

Then measure, engine by engine. The free checker on this site shows whether engines name you for up to three buyer questions. Treat it as a snapshot, not a trend.

• A blanket AI block. Disallowing every AI user agent removes ChatGPT search eligibility along with training use.
• Allowing GPTBot to get into ChatGPT. GPTBot governs training only. ChatGPT search uses OAI-SearchBot.
• Trusting robots.txt alone. A permissive file means nothing if the firewall returns 403.
• Adding schema or llms.txt as an AI fix. Google says structured data isn't required for generative AI search, and Ahrefs' matched test found AI Overview citations fell 4.6% after pages added schema. Google says Search ignores llms.txt, and no other engine documents reading it. Existing markup does no harm. It just isn't a lever for AI answers.
• Rewriting for an "AI-friendly" style. Google says you don't need to write in a specific way for generative AI search. For content advice that holds up, see how to write a post that can be cited.
• Mass-publishing AI pages. In OtterlyAI's test, Google removed both 1,000-post sites with no manual action in Search Console.
• Chasing an AI rank. Google's AI report shows impressions only. No position data exists to check a rank against.
• Hiding text for AI crawlers. Injected instructions and cloaking are manipulation. Research shows they can sway some agents, which is why engines treat them as attacks. Never use them.

Does blocking GPTBot remove my site from ChatGPT search? No. OpenAI documents GPTBot as its training crawler and OAI-SearchBot as its search crawler, and says each setting is independent of the others. You can disallow GPTBot to keep content out of training while allowing OAI-SearchBot so pages can appear in ChatGPT search answers. Blocking OAI-SearchBot is what removes you from those answers, though navigational links may still appear.

Can I block Google AI Overviews with robots.txt? No. As of October 2026, Google offers no robots.txt token specific to AI Overviews or AI Mode. Googlebot rules control all Google Search surfaces, so blocking Googlebot removes you from Search entirely. Page-level controls such as nosnippet, max-snippet and data-nosnippet limit what appears. Search Console also has a generative AI inclusion control, rolling out to a subset of site owners, where inclusion is the default.

If I rank in Google's top 10, will AI Overviews cite me? Not necessarily. Ahrefs found in March 2026 that 37.9% of URLs cited in AI Overviews also appeared in the first 10 results, so most citations came from further down. Ahrefs changed its method between studies, so don't read that as a measured decline. Ranking helps eligibility, but Google says indexing and serving aren't guaranteed.

Do I need a separate strategy for each AI engine? Partly. The access checks are shared: crawler permission, CDN reachability, HTML rendering and index status. After that, engines diverge. Google's AI features draw on Google's index, Copilot on Bing's, and ChatGPT, Perplexity and Claude each run their own crawlers. In OtterlyAI's test, one engine cited the sites thousands of times while others barely did, so measure each engine on its own.

Why is my site eligible but still not mentioned? Because eligibility only removes blocks. The engine still decides whether to search for a given question, how to rewrite it, what to retrieve and what to cite. Well-known brands dominate citations across platforms. Your next moves are stating specific facts clearly on your own pages and being named on the third-party sites engines already cite for your category.