AI SEO Optimization: What to Fix First, Ranked by Evidence
99Visibility Research · Updated October 21, 2026 · 8 min read
Short answer
AI SEO optimization works best as a list of blocks to remove, in order of evidence. Fix server-side rendering of key content first. Then fix per-bot robots.txt rules, CDN and firewall reachability, snippet controls and index status. After that, state your key facts plainly. Schema, llms.txt and AI-style rewrites sit off the list because primary sources show no reliable effect. None of this guarantees a citation.
Key takeaways
- The AI SEO fixes with strong evidence are access fixes, and they work by removing blocks rather than adding signals.
- Most AI crawlers outside Google don't run JavaScript, so facts that load only client-side may not exist for them.
- robots.txt is per bot and only half the gate: a firewall or CDN can still return a 403 to a crawler you allowed.
- Schema, llms.txt, chunking and AI-friendly rewrites have no credible evidence as levers for AI answers.
- Check index status and crawler access engine by engine, because a healthy Google tells you nothing about Bing or ChatGPT.
Why order matters: the four stages of an AI answer
Every AI answer passes through four stages, and each one has a different owner. Most AI SEO advice skips straight to the last one.
- Parametric: what the model learned in training, frozen at a cutoff. Nothing you publish this quarter changes it.
- Access: whether a crawler may fetch and store your page. You and your CDN control this.
- Retrieval: whether your page enters the model's context for a given query. The engine controls this, partly through its index.
- Generation: which retrieved pages the model uses and cites. The model controls this.
A critical survey of 45 GEO studies (arXiv, July 2026) concluded that content already retrieved can change how it gets cited. It found no reviewed technique with a stable, cross-platform effect on being discovered in the first place. Nearly every published GEO effect size was measured on pages already sitting in the model's context.
That reshapes the work. The fixes with strong evidence sit at the access stage, and they work by removing blocks. If a crawler can't read your page, nothing downstream matters. So start where the evidence is strongest and the outcome is binary.
What you control and what you don't
| Stage | What decides the outcome | In your control? |
|---|---|---|
| Parametric | Training data and the model's cutoff date | No, not this quarter |
| Access | Rendering, robots.txt, CDN rules, snippet directives | Yes |
| Retrieval | The engine's index, its rewritten queries, its ranking | Partly, through index eligibility |
| Generation | The model's choice of sources to use and cite | No |
Two things outside your site still shape the result. First, engines rewrite the user's prompt into their own search queries, so you can't optimize for the exact words a buyer types. Second, when engines cite other sites instead of yours, being named on those sites is associated with appearing in the answer. That's an association, not a promised effect.
The practical split: own the access stage completely, keep your facts clean, and accept that the model makes the final call. For a fuller breakdown, see which weaknesses in your AI search presence you can actually fix.
Fix 1: Server-render the content that matters
Most AI crawlers don't run JavaScript. Vercel's crawler study found that ChatGPT and Claude crawlers fetched JavaScript files (11.50% and 23.84% of their requests) but did not execute them, so they couldn't read client-side rendered content. Independent retesting in June 2026 found the same for GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, PerplexityBot, Meta-ExternalAgent and Bytespider. Googlebot renders JavaScript, so this fix matters most for the non-Google engines.
What to do:
- Fetch your key pages with JavaScript disabled and compare the text with the fully rendered page.
- List the facts that appear only after hydration: prices, plan names, product descriptions, headings.
- Move those into the initial HTML with server-side rendering or prerendering.
- Check tabs, accordions and "load more" sections that inject content after a click.
This fix doesn't make an engine more likely to pick you. It decides whether your content exists for that engine at all. That's why it goes first.
Fix 2 and 3: Per-bot robots.txt, then the firewall behind it
robots.txt rules apply per bot, and the bots do different jobs. As of October 2026, OpenAI's crawler documentation says each setting is independent: GPTBot governs training use, while OAI-SearchBot governs whether your site can appear in ChatGPT search answers. Perplexity uses PerplexityBot for its index, Anthropic uses Claude-SearchBot for search, and Meta uses Meta-WebIndexer for Meta AI search.
The common error is a blanket AI block copied from a blog post. It removes search eligibility along with training use. If you want to opt out of OpenAI training but stay eligible for ChatGPT search, the rules look like this:
User-agent: GPTBot
Disallow: /
User-agent: OAI-SearchBot
Allow: /
robots.txt is only half the gate. OpenAI's help center says eligibility needs OAI-SearchBot allowed and a host or CDN that lets traffic from OpenAI's published IP addresses through. A firewall or bot-management rule can return a 403 to a crawler that robots.txt allows, and no dashboard will warn you. Since 15 September 2026, Cloudflare blocks its Training and Agent crawler categories by default on ad-displaying pages for new domains, new sites on existing accounts and free-tier customers. A site can lose some AI access without anyone editing a file.
Request a page as each crawler and record the status code. The AI Crawler Checker does this and shows which crawlers your firewall turns away. When you find a block, write it up as a developer ticket that actually gets done.
Fix 4 and 5: Snippet controls and index status
Google and Bing document four on-page controls that keep content out of AI answers: noindex, nosnippet, max-snippet:0 and data-nosnippet. Bing says content marked data-nosnippet stays indexed but is excluded from snippets and AI summaries. These directives often arrive by accident, inherited from a template where someone wanted to hide a cookie banner. Search your templates and HTTP headers for them and confirm each one is intentional.
Then check index status, engine by engine. Google says a page must be indexed and eligible to show with a snippet to appear as a supporting link in AI Overviews or AI Mode, with no additional technical requirements. Bing's index backs Microsoft Copilot. One being healthy tells you nothing about the other. In Otterly's 2026 experiment, Google deindexed two AI-generated blog sites and its AI surfaces almost stopped citing them, while Bing kept them and Copilot kept citing them.
Two checks most teams skip:
- Google Search Console's generative AI control is opt-out. Inclusion is the default, but a property can be opted out, and child properties inherit the parent's setting.
- Bing Webmaster Tools is the only place that shows grounding queries, the searches an AI actually ran when it cited your pages.
Fix 6: State your key facts plainly and consistently
Once engines can reach your pages, give them something accurate to quote. Stating a specific fact on a fetchable page (a price, a number, a one-line definition of what you do) gives the engine a sentence it can lift. It's the only kind of on-page change with a positive experimental signal in the evidence.
AI platforms get specific facts wrong about specific brands. One fixable cause is a site that contradicts itself. List your core facts: pricing tiers, product names, founding year, location, integrations. Then check that the pricing page, about page, footer and any markup all say the same thing. Resolve every conflict.
Topical relevance matters too. C-SEO Bench, a 252,000-trial factorial study and the 2026 GEO survey all point to topical relevance as one of the most reproducible drivers of which retrieved page gets cited. That means covering the subject your buyers ask about in depth. It doesn't mean stuffing keyword variants: Google's AI optimization guide says you don't need to capture every long-tail variation.
Why schema, llms.txt and rewrites are off the list
These are the most sold AI SEO tactics. Primary sources don't support them as levers for AI answers.
- Schema markup. Google says structured data isn't required for generative AI search. Ahrefs' matched-control study of 1,885 pages found adding schema produced no major uplift, and AI Overview citations fell 4.6%. Those pages were already heavily cited, so whether schema helps a new page get considered is untested. Existing markup does no harm, so there's no reason to remove it.
- FAQ schema. Google deprecated FAQ rich results in May 2026.
- llms.txt. Google says Search ignores it. No other engine documents reading it.
- AI-friendly rewrites and chunking. Google's guide says you don't need to write in a specific way for generative AI search, and there's no ideal page length. C-SEO Bench found most conversational SEO methods were largely ineffective and often hurt a document's ranking.
- E-E-A-T scores and domain authority. Google says E-E-A-T isn't a specific ranking factor, so there's no score to improve. Domain Authority is a Moz metric that no engine computes.
- IndexNow pings. The specification never mentions AI, and a 200 response confirms only that the ping arrived.
The pattern is consistent. These tactics were either never tested on discovery or tested and found flat. Spend the hours on access first. If you're worried AI work could backfire, read whether optimizing for AI search can hurt your Google rankings.
Common mistakes to avoid
- Blocking every AI bot in robots.txt. It removes ChatGPT search eligibility along with training use.
- Allowing GPTBot to get into ChatGPT search. GPTBot governs training only. OAI-SearchBot governs search.
- Trusting robots.txt without testing the firewall. The CDN can overrule it silently.
- Shipping key facts only in client-side JavaScript. Most non-Google AI crawlers won't see them.
- Mass-publishing AI-generated pages. Google's spam policy targets many pages generated without adding value, and Otterly's test sites were removed with no manual action reported. See what AI tools speed up in SEO and what they don't.
- Tracking one blended score or an "AI rank". Google's AI report in Search Console shows impressions only, with no position data, and each engine behaves differently.
Measure each engine separately, and fix crawler access before you start tracking so the numbers mean something. To spread this work across a quarter, use the 30/60/90 day AI visibility plan.
Frequently asked questions
Is AI SEO optimization different from regular SEO?
Less than most vendors claim. Google says there are no additional requirements or special optimizations to appear in AI Overviews or AI Mode: the page must be indexed and snippet-eligible. Ranking still relates to citation, though loosely. Ahrefs found that 37.9% of URLs cited in AI Overviews also appeared in the first 10 results (March 2026). The AI-specific work is mostly access for non-Google engines: rendering, per-bot crawler rules and firewall settings.
Should I remove schema markup I already have?
No. Google says existing structured data does no harm. It just isn't a lever for AI answers. Ahrefs' matched test found no major citation uplift after pages added schema, and a 4.6% drop on AI Overviews. Leave working markup in place, and don't spend new hours on it expecting AI citations. Put that time into rendering, crawler access and consistent facts instead.
Can I use AI tools to produce SEO content at scale?
You can use AI to help write, but volume without value is the risk. Google says appropriate use of AI isn't against its guidelines, while scaled content abuse is. In Otterly's 2026 experiment, Google deindexed two new sites carrying about 1,000 AI-generated posts each, with no manual action reported. Copilot kept citing them through Bing, but that did not make up for losing Google.
How quickly will these fixes show up in AI answers?
Nobody can promise a timeline. Answers given without live search come from training data frozen at a cutoff, so new content reaches them only in a future model. Answers with live search depend on when each engine recrawls you and whether it searches for that question at all. Access fixes remove blocks, which is the part you control. What the models then say is up to them.
How do I check where I stand today?
Start with access. Request your pages as each AI crawler, read your robots.txt rule by rule for each bot, and review Google Search Console and Bing Webmaster Tools separately. Then check what the engines actually say about you. The free checker on this site runs up to three buyer questions and scores the answers. Treat it as a snapshot, not continuous tracking.
Sources
- The rise of the AI crawler - Vercel
- Overview of OpenAI Crawlers
- AI Features and Your Website
- Google's Guide to Optimizing for Generative AI Features on Google Search
- We Tracked 1,885 Pages Adding Schema. AI Citations Barely Moved.
- [2607.14035] Optimizing Visibility in Generative Engines: A Critical Survey of Generative Engine Optimization (2023-2026)
This page is AI-assisted. It was written from a verified evidence base and checked against our editorial policy. Engine behaviour changes often; facts are as of October 21, 2026.