AI Visibility Tracking ToolFree checker

The Future of SEO With AI: What the Dated Changes Already Show

99Visibility Research · Updated October 21, 2026 · 9 min read

Short answer

The future of SEO with AI already shows up in dated, documented changes. Since 15 September 2026, Cloudflare blocks training and agent crawlers by default on ad-displaying pages for many sites. Google retired FAQ rich results in May 2026 and says its AI search needs no special markup. First-party AI reports count impressions, not clicks. The work that holds up is access, indexing and accurate facts.

Key takeaways

  • Your AI access can change without anyone on your team editing a file, because CDN defaults moved in September 2026.
  • Google retired FAQ rich results in May 2026 and states that no special markup, AI text file or writing style is needed for its AI features.
  • Google's generative AI report in Search Console shows impressions only, so no tool can verify an AI rank or position in Google.
  • The levers with evidence behind them remove blocks: rendering, per-bot robots.txt, firewall reachability, index eligibility and snippet controls.
  • Watch dated changes from CDNs, Google, Bing and model vendors rather than predictions about where SEO is heading.

The dated shifts so far

Predictions about SEO are cheap. Dated changes from the companies that run search and AI systems are more useful, because you can check them and act on them. Here are the ones that matter most as of October 2026.

Date Shift What it means for you
February 2026 Bing Webmaster Tools launched an AI Performance report in public preview, including grounding queries You can see the queries Copilot actually ran, not just prompts someone guessed
May 2026 Google deprecated FAQ rich results, removed the docs in June and dropped Search Console API support in August FAQ markup no longer produces a Google display feature
June 2026 Google announced a generative AI report in Search Console, with impressions only You can count appearances in AI Overviews and AI Mode, not clicks
July 2026 Google's AI optimization guide said no AI text files, special markup or AI-specific writing are needed The Google path into AI answers is ordinary search eligibility
15 September 2026 Cloudflare began blocking Training and Agent crawlers by default on ad-displaying pages for new domains, new sites on existing accounts and existing free-tier customers A site can lose some AI access without editing anything

None of these say SEO is ending. They say the mechanics moved toward access and away from tricks. If you want the broader argument, read whether SEO is dead. This page sticks to what is documented.

How AI answers reach your pages

An AI answer about your brand passes through four stages. Knowing which one you are dealing with tells you what you can fix.

  1. Parametric. What the model knows with no web access, frozen at a training cutoff. OpenAI documents a 16 February 2026 cutoff for its GPT-5.6 models and 31 August 2025 for the Instant path used in ChatGPT. Nobody changes this stage this quarter.
  2. Access. Whether a crawler may fetch and store your page. You and your CDN control this.
  3. Retrieval. Whether your page enters the model's context for a given question. The engine controls this, and it rewrites the question before searching.
  4. Generation. Which retrieved pages the model uses and cites. The model controls this.

Most published GEO studies measured stage four, on pages already placed in the model's context. A July 2026 critical survey of 45 GEO studies concluded that "already-retrieved content can causally alter its citation or use, but no reviewed technique shows a stable, longitudinal, cross-platform causal effect on organic discoverability." That is the honest frame for the future of SEO: the proven levers sit at stage two. For a wider look at what changed, see the impact of AI on SEO.

Access can now change without you

The Cloudflare change is the clearest sign of where things are heading. From 15 September 2026, Cloudflare blocks Training and Agent crawler categories by default on pages that display ads, for the groups listed above, while Search stays allowed by default. Your robots.txt can say yes while your CDN says no, and nothing in your analytics will flag it.

Three controls decide AI access, and each works per bot:

  • robots.txt rules are per crawler. OpenAI states that "each setting is independent of the others": GPTBot governs training use, OAI-SearchBot governs ChatGPT search.
  • The firewall or CDN is a second gate. OpenAI's help centre says eligibility means allowing OAI-SearchBot and confirming your host or CDN accepts traffic from its published IP ranges. A 403 to that bot is a silent failure.
  • Some user-triggered fetchers behave differently. Perplexity documents that Perplexity-User generally ignores robots.txt. The Perplexity visibility factors page covers that split.

A blanket AI block is the common error. This snippet blocks training while keeping ChatGPT search eligibility:

User-agent: GPTBot
Disallow: /

User-agent: OAI-SearchBot
Allow: /

To see what each crawler actually gets back, the AI Crawler Checker requests a page as each AI bot and shows which ones your firewall turns away.

Markup, files and formats: what Google retired or rejected

A lot of AI SEO advice from 2024 and 2025 has lost its footing in primary sources.

  • FAQ schema. FAQ rich results were deprecated in May 2026 and the documentation removed in June. Google says structured data isn't required for generative AI search.
  • Schema for citations. Ahrefs tracked 1,885 pages adding schema against matched controls and found AI Overview citations fell 4.6%, with no major uplift on any platform. Every page in that study was already heavily cited, so whether schema helps a page enter the answer at all remains untested.
  • llms.txt. Google's AI optimization guide says Google Search ignores AI text files. No other engine documents reading one.
  • Chunking and AI-friendly writing. The same guide says you don't need to write a specific way for AI search and that there is no ideal page length.

Don't overcorrect. Google says existing markup does no harm, so ripping it out is not a fix either. The accurate line is that markup is not a lever for AI answers. One on-page signal does have some support: stating a specific fact, such as a price or a one-line definition, on a page crawlers can fetch gives an engine a sentence it can quote.

Measurement: impressions without clicks

The future of SEO measurement is narrower than most dashboards suggest.

Google

Search Console's generative AI report shows impressions for AI Overviews and AI Mode, broken down by page, country, date and device. It shows no clicks, no CTR and no position. Google has described more metrics only as a future intention. A Google-side bug depressed impressions in both the Search and generative AI reports from around 12 August 2026, so treat trends across that window with care.

The same area holds a control most owners never check. Inclusion in Google's generative features is opt-out, on by default, and child properties inherit their parent's setting. A property can be excluded by inheritance and you would not see it from outside.

Bing

Bing's AI Performance report exposes grounding queries, the phrases the AI used when retrieving cited content. As of August 2026 there was no API for it. Microsoft frames Citation Share as an observational metric, not a ranking.

Rank

Ahrefs found that 37.9% of URLs cited in AI Overviews also appeared in the first 10 results (March 2026). Ahrefs changed its method between studies, so don't read an older, higher figure as a measured decline. Since Google exposes no position data for AI features, any "AI rank" in Google has nothing to be checked against.

What is in your control, and what is not

Split the work honestly and you stop paying for effort that cannot move anything.

In your control:

  • Server-side rendering of the content that matters. Vercel found that ChatGPT and Claude crawlers fetch JavaScript files but don't execute them. Googlebot renders; most AI crawlers don't.
  • robots.txt rules per bot, and CDN or firewall settings per bot.
  • noindex, nosnippet, max-snippet and data-nosnippet, which keep content out of AI answers if applied, including by accident through a template.
  • Index status in Google and Bing, checked separately.
  • Consistent facts across your own pages. If pricing reads differently on three pages, an engine has three answers to pick from. See what to do when ChatGPT gets your pricing wrong.

Not in your control:

  • What the model memorised before its cutoff.
  • How the engine rewrites the question and which pages it retrieves.
  • Where your page sits in the model's context, and which sources it cites.
  • Brand prominence. Large brands dominate AI citations, and no page edit changes that this quarter. Being named on sites the engines already cite is associated with appearing in answers, with no promised effect.

A practical order of work and common mistakes

Order of work

  1. Test access per bot. Read robots.txt for OAI-SearchBot, PerplexityBot, Claude-SearchBot, Meta-WebIndexer and Googlebot. Then request a page as each one and confirm a 200, not a 403 or a challenge page.
  2. Check your CDN defaults. If you use Cloudflare, review which AI crawler categories it now blocks.
  3. Diff raw HTML against the rendered page. Anything that only appears after JavaScript runs, such as prices or plan names, needs to move into the server response.
  4. Search for accidental suppression. Look for noindex, nosnippet and data-nosnippet in templates, and check the generative AI setting in Search Console.
  5. Confirm index status in both Google and Bing.
  6. Fix your facts. Make each price, product name and key fact identical everywhere you state it. After a rename or relaunch, the rebrand checklist for AI answers sets out the order.

Common mistakes

  • Allowing GPTBot to "get into ChatGPT". GPTBot governs training only.
  • Treating one index as all of them. In Otterly's 2026 experiment, Google deindexed two AI-generated test sites while Bing kept them, and Microsoft Copilot kept citing them.
  • Mass-publishing thin pages. Google's spam policy names generating many pages with AI tools without adding value as scaled content abuse.
  • Reporting one blended AI score. Engines differ too much for an average to tell you what to fix.

What to monitor from here

The useful habit is watching dated changes from the operators, not commentary about them. Keep a short list and check it monthly.

  • Your CDN's bot settings. Defaults can move again, and they apply to your zone whether you read the announcement or not.
  • Vendor crawler pages. OpenAI, Anthropic, Perplexity, Google and Meta each document their bots. New user agents and changed purposes appear there first.
  • Search Console. Watch for new metrics in the generative AI report and for the inclusion control reaching your property.
  • Bing Webmaster Tools. Export grounding queries now, and watch for the API Microsoft has indicated is coming.
  • Model cutoffs. Each new model moves the parametric boundary. Content published after a cutoff is absent from that model's answers when it doesn't search.
  • What each engine says about you. Check engine by engine, for example with a free check of what Copilot says about you. The free checker on this site gives a snapshot for up to three buyer questions, which is a fair starting baseline.

For the parts of classic SEO that still carry weight, read SEO in the age of AI.

Frequently asked questions

Will traditional SEO still matter as AI answers grow?

Yes, at least for Google. Google says a page must be indexed and eligible to show with a snippet to appear as a supporting link in AI Overviews or AI Mode, and that no extra requirements exist. Crawlability, indexing and clear content are still the entry ticket. What has faded is the case for AI-specific tricks such as special markup, AI text files or rewriting pages in a particular style.

Should I remove the schema markup I already have?

No. Google says existing structured data does no harm. The evidence simply doesn't support it as a way into AI answers: Ahrefs' matched test found AI Overview citations fell 4.6% after pages added schema, and FAQ rich results were retired in May 2026. Keep what you have if it serves other purposes, and don't spend new effort on markup expecting AI citations.

Should I block AI crawlers?

Decide per bot, because the consequences differ. Blocking GPTBot stops OpenAI training use, while blocking OAI-SearchBot removes you from ChatGPT search answers. OpenAI states each setting is independent. A blanket AI block is the most common error. After any change, test that your CDN or firewall matches your robots.txt, since either one can turn a crawler away.

Can I track my ranking position inside Google's AI answers?

Not in a way anyone can verify. Google's generative AI report in Search Console shows impressions only, with no clicks, CTR or position. Google also states that third-party tools don't have access to its internal ranking data. Presence or absence of your brand for specific questions, checked per engine, is a more honest measure than any claimed AI rank.

Does updating page dates help AI engines cite me?

The evidence doesn't show that. Anthropic's search tool exposes a page age field, but no vendor says it influences selection. The main study linking freshness to citation had no comparison group of uncited pages. Dates are self-declared, so bumping them proves nothing. Update pages when the facts change, and make sure the new facts appear in the HTML.

Sources

  1. AI Features and Your Website
  2. Google's Guide to Optimizing for Generative AI Features on Google Search
  3. Overview of OpenAI Crawlers
  4. We Tracked 1,885 Pages Adding Schema. AI Citations Barely Moved.
  5. [2607.14035] Optimizing Visibility in Generative Engines: A Critical Survey of Generative Engine Optimization (2023-2026)
  6. 2000 AI Blogs Experiment: Does Google Penalize AI Content?

This page is AI-assisted. It was written from a verified evidence base and checked against our editorial policy. Engine behaviour changes often; facts are as of October 21, 2026.

Keep reading