SEO for AI Search: What Actually Gets You Into Answers

Every AI answer comes from one of four stages. Mixing them up is where most bad advice starts.

Retrieval is not guaranteed. As of October 2026, ChatGPT decides per query whether to search, Google AI Overviews often don't trigger, and search in Claude.ai is a user toggle. Some answers never touch the live web at all. Those come from what the model learned in training, which is a separate problem covered in training data vs live retrieval.

What this means: a page has to clear access before anything else can happen. If a crawler can't read it, no retrieval system can pick it up and no model can cite it.

You control access. You don't control the rest, and nobody outside the engines can see it.

• The real queries are hidden. OpenAI's help center says ChatGPT search rewrites a prompt into one or more targeted queries. Google may fan a question out across subtopics. The prompts you test are a proxy for queries you can't observe.
• Position in the context window is internal. C-SEO Bench (NeurIPS 2025) found that a document's rank inside the model's context mattered far more than rewriting the document. No publisher can set that rank.
• Citation choice belongs to the model. It decides which retrieved pages to quote.
• Training data is fixed until the next model. Nothing you publish today reaches an answer given with search off.

The research is blunt about content tactics. A July 2026 critical survey of 45 GEO studies (arXiv 2607.14035) concluded that already-retrieved content can change how it gets cited, but no reviewed technique shows a stable, cross-platform causal effect on whether a page gets discovered. Almost every published GEO effect was measured on pages already sitting in the model's context.

The next move: spend your effort on the stage you own, then measure the rest honestly.

These four checks have a documented mechanism, stated by the engines or crawler operators themselves. They work by removing blocks, not by adding signals.

Server-render the content that matters

Vercel's crawler study found that ChatGPT and Claude crawlers fetch JavaScript files (11.50% and 23.84% of requests respectively) but don't execute them, so they can't read client-side rendered content. Later testing reported in June 2026 found the same for OAI-SearchBot, Claude-SearchBot, PerplexityBot and others. Googlebot renders JavaScript, so this matters mainly for non-Google engines. Put prices, product names, definitions and key claims in the initial HTML.

Set robots.txt per bot

OpenAI states that each of its crawler settings is independent. GPTBot controls training use. OAI-SearchBot controls whether you appear in ChatGPT search answers. A site can block one and allow the other:

User-agent: GPTBot
Disallow: /

User-agent: OAI-SearchBot
Allow: /

Do the same check for PerplexityBot, Claude-SearchBot, Meta-WebIndexer and bingbot. Meta is the one operator that says allowing its indexer helps it cite and link to your content. User-triggered fetchers such as ChatGPT-User and Perplexity-User may not follow robots.txt, so treat it as necessary, not sufficient. More detail on OpenAI's side is in what you control for ChatGPT citations.

Make sure your CDN or firewall lets them in

OpenAI's help center says eligibility needs both things: allow OAI-SearchBot in robots.txt and confirm your host or CDN accepts traffic from its published IP ranges. Bot protection often returns a 403 to legitimate crawlers. From 15 September 2026, Cloudflare blocks Training and Agent crawler categories by default on ad-displaying pages for new domains and existing free-tier customers, while Search stays allowed. A site can lose access without anyone editing it. The AI Crawler Checker requests a page as each AI crawler and shows which ones your firewall turns away.

Stay indexed and snippet-eligible

Google states that a page must be indexed and eligible to show with a snippet to appear as a supporting link in AI Overviews or AI Mode, with no additional technical requirements. There is no robots.txt token for AI Overviews. Search Console has a generative AI control that is on by default and can inherit from a parent property, so check it. On the page, noindex, nosnippet, max-snippet:0 and data-nosnippet keep content out of AI answers on Google, and Bing added data-nosnippet support in October 2025. Copilot grounds on the Bing index, covered in getting into Microsoft's AI answers.

Much of the AI search advice in circulation is contradicted by primary sources. Here is what the evidence actually says.

• Schema markup. Google says structured data isn't required for generative AI search. Ahrefs tracked 1,885 pages that added schema against matched controls and found AI Overview citations fell 4.6%. Existing markup does no harm, so there's no reason to rip it out. It just isn't a lever for AI answers. FAQ rich results were deprecated in May 2026.
• llms.txt. Google says Search ignores it. No other engine documents reading it.
• Chunking and AI-friendly writing. Google's AI optimization guide says you don't need to write in a special way for generative AI and that there's no ideal page length.
• E-E-A-T scores. Google says E-E-A-T isn't a specific ranking factor. There is no score to raise.
• Domain authority. It's a Moz metric. No engine computes it.
• IndexNow pings. The specification never mentions AI. A 200 response only confirms the ping arrived.
• "Up to 40% more visibility." That figure comes from a lab engine where the sources were already loaded into context.

If a vendor promises citations from any of these, ask what study shows it on a live engine. For how to approach writing itself, see how to write a blog post for AI citations.

Content isn't irrelevant. The evidence supports a narrower role than the market claims.

• State specific facts plainly. A price, a number or a one-line definition on a fetchable page gives an engine a sentence it can quote. That's the only positive experimental signal for on-page content. It also helps engines get your facts right, which is why publishing your prices is worth considering.
• Be topically relevant. Multiple studies agree that topical fit to the query drives citation. This is fit, not keyword coverage. The original GEO paper found keyword stuffing lowered its own visibility metric.
• Get named on the sites engines already cite. When answers cite third-party pages instead of yours, being mentioned there is the route in. This is a strong association, not a promised effect.
• Classic search still counts on Google. Ahrefs found in March 2026 that 37.9% of URLs cited in AI Overviews also appeared in the first 10 results. Ranking helps, but most citations come from elsewhere. The Google-specific steps are in optimizing for AI Overviews.

Never use hidden text, cloaking for AI crawlers or injected instructions. Research shows manipulated content can sway some search agents, but it's illegitimate, it violates engine policies, and it puts your brand at risk.

Work from the hardest blocks to the softest. Each step is checkable.

1. Read your robots.txt per bot. Confirm OAI-SearchBot, PerplexityBot, Claude-SearchBot, Meta-WebIndexer, bingbot and Googlebot are allowed. Decide training bots (GPTBot, ClaudeBot, Google-Extended) separately.
2. Test reachability at the network layer. Request key pages with each crawler's user agent and look for 403s, 429s and challenge pages. If you're on Cloudflare, review the AI crawler categories after the September 2026 change.
3. Diff raw HTML against the rendered page. Do this for pricing, product, about and top category pages. Move anything that only appears after JavaScript into the server response.
4. Audit suppression controls. Search templates for noindex, nosnippet, max-snippet and data-nosnippet. Check the Search Console generative AI control and confirm index status in both Search Console and Bing Webmaster Tools.
5. Make your key facts consistent. Your price, founding year and product names should match across pages, footer and markup.
6. Measure each engine separately, logged out. Memory and personalization change answers, as explained in logged-in vs logged-out testing. The free checker on this site gives a snapshot of whether AI names you for three buyer questions.

• A blanket AI block. A single rule that blocks every AI agent silently removes ChatGPT search eligibility along with training.
• Allowing GPTBot to get found. GPTBot is for training only. It does nothing for ChatGPT search.
• Trusting robots.txt alone. A firewall returning 403 to OAI-SearchBot cancels an Allow rule completely.
• Prices loaded by JavaScript. Most non-Google crawlers will see an empty price table.
• Inherited suppression. A data-nosnippet added to hide a cookie banner can spread across a template and hide real content.
• Chasing an "AI rank". Google's Search Console AI report shows impressions only, with no position data, so no tool can validate a rank in AI Overviews.
• Trusting one blended score. Engines use different indexes and models. A single number hides an engine that never cites you.