How AI Is Changing SEO, Change by Change

Classic SEO had one main question: does Google rank this page? AI answers add more steps, and each step has a different owner.

• Parametric knowledge. What a model knows with no web access, frozen at a training cutoff. Nobody changes this quickly.
• Access. Whether a crawler may fetch and store your page. You and your CDN control this.
• Retrieval. Whether your page enters the model's context for a given question. The engine controls this, partly through its index.
• Generation. Which retrieved pages get used and cited. The model controls this.

The research matters here. A 2026 critical survey of 45 GEO studies (arXiv 2607.14035) found that already-retrieved content can change how it gets cited, but no technique showed a stable, cross-platform effect on being discovered in the first place. Most "GEO tactics" were tested on pages already inside the model's context.

So the practical change to SEO is narrower than the hype. Ranking and helpful content still matter. What's new is a set of access gates, per engine, that can block you silently. If you want the broader argument, see whether SEO is dead.

These are the changes with a primary source and a date behind them, as of October 2026.

The Cloudflare change is the one most likely to hit a site without a deploy. Search crawlers stay allowed by default, but agent fetches can be blocked. Read how crawler access can break without you changing anything.

Each AI vendor now documents several bots, and each does a different job. OpenAI's crawler page says GPTBot is for training, OAI-SearchBot surfaces sites in ChatGPT search, and "each setting is independent of the others." Anthropic separates ClaudeBot (training) from Claude-SearchBot (search quality). Perplexity says PerplexityBot is for search indexing, not model training.

This is where most real damage happens. A site that copied a blanket "block all AI bots" rule may have removed itself from ChatGPT search while only meaning to opt out of training. OpenAI states that sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though they can still appear as navigational links.

A rule that opts out of training but stays in ChatGPT search looks like this:

User-agent: GPTBot
Disallow: /

User-agent: OAI-SearchBot
Allow: /

Two caveats. User-triggered fetchers behave differently: OpenAI says robots.txt "may not apply" to ChatGPT-User, and Perplexity says Perplexity-User generally ignores robots.txt. And there is no robots.txt token for Google AI Overviews or AI Mode. Googlebot rules control those, and Google-Extended controls Gemini training and grounding without affecting Search.

What to do: list every bot in your robots.txt, decide training and search separately, then confirm your firewall lets the allowed ones through. OpenAI's help center says eligibility needs both robots.txt permission and a host or CDN that allows its published search bot IP addresses.

On 2026-07-10 Google published its guide to optimizing for generative AI features in Search. It is the clearest first-party statement of what does not matter for Google's AI surfaces.

• No AI text files. Google says you don't need new machine-readable files, AI text files, markup or Markdown, because Google Search doesn't use them. That covers llms.txt for Google. No other engine documents reading it either.
• No AI-specific writing. Google says you don't need to write in a specific way for generative AI search, or chase long-tail keyword variants.
• No chunking requirement. Google rejects chunking as a requirement and says there is no ideal page length.

Google's AI features page adds the eligibility rule: a page must be indexed and eligible to show with a snippet, and "there are no additional technical requirements." Inclusion is not guaranteed.

Schema deserves its own line. Google says structured data isn't required for generative AI search. Ahrefs' matched-control study of 1,885 pages found no major citation uplift on any platform after adding schema, and a 4.6% drop on AI Overviews. Ahrefs only studied pages already heavily cited, so the effect on pages trying to break in is untested. Existing markup does no harm. It just isn't a lever for AI answers.

What to do: stop spending on AI-only formatting projects. Put that time into index health and snippet eligibility. The generative AI and SEO overview covers what still carries over from classic SEO.

Search Console's generative AI report

Google announced a generative AI performance report on 2026-06-03, and it reached all properties around 2026-08-11. It shows impressions for AI Overviews and AI Mode, split by page, country, date and device. It has no clicks, no CTR and no position. The Search Analytics API does not cover it. A Google-side bug depressed impressions from about 2026-08-12, so treat trends from that window with caution.

The same period brought an inclusion control. It is opt-out: inclusion is the default, and child properties inherit from the parent. A property can be excluded from AI Overviews, AI Mode and Discover's generative features without anyone noticing.

What to do: check the inclusion control on every property, including child properties. Read the report as a presence signal, not a traffic signal. Any tool that shows your average position in AI Overviews has no Google data to check it against.

Bing's AI Performance report

Bing Webmaster Tools launched AI Performance in public preview on 2026-02-10 and expanded it on 2026-06-16. It covers Microsoft Copilot, Bing AI summaries and some partner integrations. It shows total citations, cited pages, intents, topics, Citation Share, and grounding queries: the phrases the AI used when retrieving your content.

Grounding queries matter because engines rewrite user prompts before searching. OpenAI says ChatGPT search typically rewrites your query into one or more targeted queries. Bing's report is the only place that exposes those real retrieval inputs. It has no API as of October 2026, and Microsoft says grounding query data is a sample.

What to do: verify your site in Bing Webmaster Tools and export grounding queries each month. Microsoft calls Citation Share an observational metric, not a ranking. Treat it that way.

Be plain with yourself and your team about this split.

In your control:

• Per-bot robots.txt rules.
• Firewall and CDN settings that decide whether a crawler gets a 200 or a 403.
• Server-side rendering of the content that matters. Vercel's crawler study found ChatGPT and Claude crawlers fetch JavaScript files but don't execute them, so content that appears only after JavaScript runs may be invisible to them. Googlebot renders JavaScript.
• Index and snippet controls: noindex, nosnippet, max-snippet and data-nosnippet keep content out of AI answers, on purpose or by accident.
• Stating specific facts (prices, numbers, one-line definitions) clearly on fetchable pages, so an engine has a sentence it can quote.

Not in your control:

• Whether an engine searches the web for a given question at all. Retrieval is discretionary per query.
• Where your page sits in the model's candidate list, and which retrieved pages it cites.
• What a model learned before its training cutoff.

When engines cite other sites instead of yours, being named on those sites is associated with getting into the answer. That is an association, not a promised effect.

Work from the gates outward. Each step only matters if the one before it passes.

1. Audit robots.txt per bot. Separate training bots (GPTBot, ClaudeBot, Google-Extended) from search bots (OAI-SearchBot, Claude-SearchBot, PerplexityBot, Meta-WebIndexer). Meta states that allowing Meta-WebIndexer helps it cite and link to your content.
2. Test real reachability. Request key pages as each crawler and look for 403s, challenges or CAPTCHAs. The AI Crawler Checker does this and shows which crawlers your firewall turns away.
3. Check rendering. Compare raw HTML with the rendered page. Move prices, product names and key claims into the initial HTML.
4. Check index status in both engines. Use Search Console and Bing Webmaster Tools separately. One healthy index tells you nothing about the other. Here is how to tell if Google has deindexed your site.
5. Check suppression. Look for inherited nosnippet or data-nosnippet tags and the Search Console AI inclusion control.
6. Fix your own facts. Make pricing, founding details and product names consistent across every page.
7. Re-check on a schedule. Access breaks without deploys. See what changes between AI visibility checks.

• Allowing GPTBot to get into ChatGPT. GPTBot governs training only. OAI-SearchBot governs ChatGPT search.
• Assuming blocking GPTBot hides you from ChatGPT search. It doesn't. The settings are independent.
• Trusting robots.txt alone. A CDN or firewall can block a crawler that robots.txt allows.
• Buying an "AI rank" or E-E-A-T score. Google says E-E-A-T isn't a specific ranking factor, and its AI report has no position data.
• Bumping the last-updated date. The date is self-declared, and no engine says it weights it. See whether changing the last-updated date helps.
• Quoting "up to 40% more visibility." That figure came from a lab engine with sources preloaded into context.
• Hiding text for AI crawlers. Cloaking and prompt injection are illegitimate. Research on search agents shows manipulated content can sway answers, which is exactly why engines and Google's spam policies treat it as abuse.