AI Search Optimization vs SEO: Where Effort Should Go

Classic SEO and AI search optimization are not two separate budgets competing for the same money. For Google, they are the same job. Google's AI features documentation says a page must be indexed and eligible to be shown with a snippet to appear as a supporting link in AI Overviews or AI Mode. It adds: "There are no additional technical requirements."

Outside Google, the picture changes. ChatGPT, Perplexity, Claude and Meta AI each run their own crawlers. Microsoft Copilot grounds its answers in the Bing index. A site can be perfectly healthy in Google and still be unreachable to every one of them.

So the honest split looks like this:

• Shared work: indexable pages, snippet eligibility, content that matches what buyers ask. This is search engine optimization as you already know it.
• AI-specific work: per-bot robots.txt rules, firewall and CDN reachability, server-side rendering of key facts, and Bing index presence.
• Not a lever: most of what is sold as generative engine optimization, such as AI-specific markup, llms.txt files or rewriting pages in an "AI style".

The next move is to stop thinking "SEO or AI" and start thinking "which gates does each engine need my site to pass".

Every AI answer that cites the web goes through the same chain. A crawler must be allowed to fetch the page. The page must land in an index. The engine must retrieve it for a specific question. Then the model decides whether to use and cite it. The first gate differs by engine, and that is where SEO and AI work diverge.

As of October 2026, the documented access controls look like this:

Two details trip teams up. First, OpenAI's crawler page says each bot setting "is independent of the others". GPTBot governs training use only. OAI-SearchBot governs whether you appear in ChatGPT search answers. Second, OpenAI's help center says eligibility requires allowing OAI-SearchBot and confirming your host or CDN allows traffic from OpenAI's published search bot IP addresses. A robots.txt allow means nothing if your firewall returns a 403.

A robots.txt that opts out of training but stays in ChatGPT search looks like this:

User-agent: OAI-SearchBot
Allow: /

User-agent: GPTBot
Disallow: /

The takeaway: your Google work carries over to Google's AI features. Every other engine needs its own gate checked.

It helps to separate four stages, because each has a different owner.

1. Parametric knowledge. What a model knows with no web access is frozen at its training cutoff. Nothing you publish this quarter reaches those answers.
2. Access. Whether a crawler can fetch and store your page. This is yours: robots.txt, firewall rules, rendering, noindex and snippet controls.
3. Retrieval. Whether your page is pulled in for a given question. You influence this only through topical relevance and being indexed. Engines rewrite the user's question into their own searches, which nobody outside can see.
4. Generation. Which retrieved pages the model uses and cites. This belongs to the model.

The evidence lines up with that split. A July 2026 critical survey of 45 GEO studies (arXiv 2607.14035) concluded that already-retrieved content can change its citation or use, but no reviewed technique shows a stable, longitudinal, cross-platform causal effect on discoverability. Most headline GEO numbers come from lab setups where the page was already placed in the model's context. The gap between those setups and real engines is covered in lab GEO studies vs live engines.

What that means for budget: the access stage is where your money buys a documented outcome, which is eligibility. It does not buy a citation. For a fuller map of the limits, see what you can and can't influence in LLM optimization.

Say you run a B2B software company with one SEO contractor and a small content team. Here is an order that puts the cheapest, best-evidenced fixes first.

1. Confirm Google eligibility

Check that key pages are indexed and not carrying noindex, nosnippet or max-snippet:0 by accident. Check data-nosnippet too, which Bing and Google both honor for snippets and AI summaries. These often come from a template. This step serves classic SEO and Google's AI features at once.

2. Audit per-bot access

Read your robots.txt bot by bot. Then request a page as each AI crawler and see what comes back. The AI Crawler Checker does this and shows which crawlers your firewall turns away. A common failure is a blanket AI block that removes ChatGPT search eligibility along with training.

3. Render the facts that matter in HTML

Vercel's crawler study found ChatGPT and Claude crawlers fetch JavaScript files but do not execute them. Googlebot does render. If your pricing, product names or key claims appear only after JavaScript runs, Google sees them and most AI crawlers do not. Move those facts into the server-rendered HTML.

4. Check Bing separately

Copilot grounds on Bing. Verify the site in Bing Webmaster Tools and confirm your pages are indexed there. A healthy Google index tells you nothing about Bing.

5. Then invest in content

Write pages that answer the questions buyers actually ask, and state specific facts plainly: a price, a number, a one-line definition. A fetchable sentence gives an engine something to quote. This is ordinary good content work, not a separate AI discipline.

Steps 1 to 4 are mostly one-off and checkable. Step 5 is ongoing and is where most of your existing SEO budget already sits.

Rankings and AI citations are related but not the same thing. Ahrefs found in March 2026 that 37.9% of URLs cited in AI Overviews also appeared in the first 10 results. That means most cited URLs did not come from the top 10. Ahrefs changed its method between studies, so treat that figure as a snapshot, not proof of a decline.

The index itself matters more than the rank. Otterly's 2026 experiment published about 2,000 AI-generated posts across two new domains. Google deindexed both sites, and Google's AI surfaces almost stopped citing them. Bing kept the sites, and Microsoft Copilot accounted for 2,718 of the 3,934 citations Otterly tracked for one of them. ChatGPT barely cited either site, even while they ranked in Bing. It is one pair of sites on a low-stakes topic, so read it as an illustration, not a law.

Two lessons follow. Losing Google's index removes you from Google's AI features. Being indexed somewhere is necessary but not enough for any particular engine. Neither lesson justifies shortcuts: mass-publishing thin pages is exactly the scaled content abuse Google's spam policy describes. If you're worried that AI-focused changes could cost you organic traffic, see whether AI optimization can hurt Google rankings.

• Allowing GPTBot to get into ChatGPT. GPTBot is a training crawler. ChatGPT search uses OAI-SearchBot. Blocking GPTBot does not remove you from ChatGPT search either.
• Trusting robots.txt alone. A CDN or firewall can block a crawler your robots.txt allows. Perplexity also documents that its user-triggered fetcher generally ignores robots.txt rules, so treat robots.txt as necessary, not sufficient.
• Paying for markup as an AI lever. Google says structured data isn't required for generative AI search. Ahrefs' matched test found AI Overview citations fell 4.6% after pages added schema. Existing markup does no harm, but it is not a reason to expect citations.
• Publishing llms.txt or "AI-friendly" rewrites. Google's AI optimization guide says Google Search ignores AI text files and that you don't need to write in a specific way for generative AI search. No other engine documents reading llms.txt.
• Chasing scores that don't exist. There is no E-E-A-T score and no engine computes domain authority. Google does run a news-scoped topic authority system, which is different from any third-party metric.
• Hiring two agencies for one job. If your SEO provider already handles indexing and content, the AI layer is mostly an access audit. Ask them about it first, as covered in whether your SEO agency should already handle AI search.

Measurement is where AI work is weakest, so set expectations before you spend.

• Google: Search Console's generative AI report shows impressions for AI Overviews and AI Mode. As of October 2026 it shows no clicks, no click-through rate and no position. There is no ground truth for an "AI rank".
• Bing: Bing Webmaster Tools' AI Performance report shows citations and the grounding queries Copilot actually ran. It is the only first-party view of real retrieval queries.
• Prompt trackers: these sample answers to prompts someone chose. In Otterly's test, Microsoft's own report counted 42,817 citations for one site while Otterly's tracker logged 3,934. A sample is a sample.

Score each engine on its own. A blended number would have hidden that one engine cited Otterly's sites thousands of times while others barely did. For a quick snapshot, the free checker on this site shows whether AI engines name you for up to three buyer questions. Treat it as a starting point, then fix whatever access gaps it points to.

Does ranking in Google's top 10 get me into AI Overviews? It helps but guarantees nothing. Ahrefs found in March 2026 that 37.9% of URLs cited in AI Overviews also ranked in the first 10 results, so most citations came from elsewhere. Google's documented requirement is only that a page is indexed and eligible for a snippet. After that, Google says indexing and serving aren't guaranteed. Keep doing solid SEO, but don't treat a ranking as a citation.

Should I block GPTBot if I want to appear in ChatGPT? That is a separate decision. GPTBot controls whether OpenAI may use your content for training. OAI-SearchBot controls whether your pages can appear in ChatGPT search answers. OpenAI says each setting is independent, so you can block GPTBot and still allow OAI-SearchBot. Whatever you choose, confirm your CDN or firewall isn't blocking OAI-SearchBot, because robots.txt alone isn't enough.

Is there anything I need to do specifically for Google AI Mode? Nothing beyond normal Search eligibility, according to Google's AI features documentation. A page that is indexed and snippet-eligible can appear as a supporting link. There is no robots.txt token just for AI Overviews or AI Mode. The controls are Googlebot rules and the noindex and snippet directives. Search Console is rolling out an opt-out for generative AI features, and inclusion is the default.

Do I need a separate budget line for AI search? Usually a small one. The AI-specific work is mostly a one-off access audit: per-bot robots.txt rules, firewall reachability, server-side rendering of key facts and Bing indexing. Content and indexing work already sits in your SEO budget and serves Google's AI features too. Be wary of retainers that sell markup, llms.txt or AI-style rewrites as ways to get cited, because the evidence doesn't support them.

Why do AI crawlers miss content Google can see? Rendering. Googlebot executes JavaScript, but Vercel's crawler study found ChatGPT and Claude crawlers fetch JavaScript files without running them. Content injected by client-side code, such as prices loaded after the page renders, can be missing from what those crawlers store. Compare the raw HTML with the rendered page. Any important fact that appears only after rendering should move into the server-rendered HTML.