Rolling Out SSR or Prerendering for Key Pages

AI crawlers read the HTML your server sends. Most of them stop there. Vercel's study of AI crawler traffic found that ChatGPT and Claude crawlers fetched JavaScript files (11.50% and 23.84% of their requests) but did not execute them, so they could not read client-side rendered content. Independent re-testing reported in June 2026 found the same for GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, PerplexityBot, Meta-ExternalAgent and Bytespider. The Vercel study breakdown covers the method and its limits.

Googlebot is the exception: it renders JavaScript. So this fix matters for ChatGPT, Claude, Perplexity and Meta, and much less for Google's own surfaces. See whether AI Overviews need server-side rendering for the Google side.

Be clear about the outcome. Server-rendering decides whether your content exists for a crawler at all. It does not make an engine more likely to pick you once it can read you. That is why we call it hygiene, not optimization, and why the honest answer to whether SSR increases AI citations is that it removes a block, nothing more.

You have three ways to get text into the first response: server-side rendering on each request, prerendering at build or on first request, or full static generation. The SSR vs prerendering vs static comparison helps you pick. This playbook works for any of them.

Don't migrate the whole site at once. Rank templates by how much damage a missing fact does.

Two things move a template up the list. First, content loaded only after a click: tabs, accordions, "load more" and infinite scroll that fetch text after interaction. Second, facts pulled from an API at runtime, such as a pricing table fed by a billing service. Before you plan, run the find facts hidden behind JavaScript audit so you know which templates actually have a gap.

1. List the load-bearing facts per template. For each template, write down the five to ten sentences that must reach a crawler: price, plan names, what the product does, location, key integrations.
2. Measure the gap before you change anything. Fetch each template's raw HTML with no JavaScript, then the rendered DOM, and diff the visible text. The raw HTML vs rendered DOM check walks through this. Save the results as your baseline.
3. Pick the rendering mode per template. Pages that rarely change (about, docs) suit static generation. Pages with frequent updates (pricing tied to a billing system) suit server rendering or prerendering with a short cache.
4. Ship one template behind a normal deploy. Start with pricing. Don't add a crawler-only branch. The same HTML should go to every visitor.
5. Move interaction-gated text into the initial HTML. Accordion and tab content can stay collapsed visually, as long as the text is in the markup on first response.
6. Verify, then move to the next template. Run the checklist below. Only proceed when the current template passes.
7. Check the side effects. Confirm that no template picked up noindex, nosnippet or data-nosnippet during the migration, and that raw HTML stays well under Googlebot's 2MB per-URL fetch limit. Most pages never get close.

The tempting shortcut is to detect AI crawler user-agents and send them a special page. Don't. If that page says something different from what users see, it is cloaking. Google's spam policies treat cloaking as a violation, and the same pattern aimed at AI crawlers is a form of manipulation this library treats as illegitimate.

The risk is concrete. A crawler-only page can drift from the real one, so engines learn facts your customers can't verify. It can trigger spam action in Google. And it puts you in the same category as hidden text and prompt injection, which research on LLM search agents studies as attacks, not tactics.

The safe rule is simple: prerendering changes when the HTML is built, never what it says. If you use a prerender service that serves cached HTML to bots, check that the cached version matches the live page word for word on the facts that matter. Keep the cache short enough that a price change reaches both versions on the same day.

Run this for each template after it ships:

• Fetch the URL with JavaScript disabled (a plain curl request works) and confirm every load-bearing fact from step 1 appears in the response.
• Repeat the fetch using the published user-agent strings for OAI-SearchBot, PerplexityBot and Claude-SearchBot (OpenAI, Perplexity and Anthropic list them on their crawler pages).
• Record the status code for each crawler fetch. A 200 is a pass. A 403, 429 or a challenge page means your CDN or firewall is turning the crawler away, and rendering won't help until that is fixed.
• Diff the bot fetch against a normal browser fetch. The facts must match.
• Check the response for noindex, nosnippet, max-snippet:0 and data-nosnippet on content you want quoted.
• Confirm /robots.txt still allows the retrieval bots you want, and that it is itself reachable.
• Spot-check one interaction-gated block (a tab or accordion) to confirm its text is in the markup.

A quick manual test looks like this:

curl -s https://example.com/pricing | grep -i "per month"

If you'd rather not script it, the free JavaScript rendering check compares raw and rendered text, and 99Visibility's AI Crawler Checker requests a page as each AI crawler and shows which ones your firewall turns away.

Good

• Every listed fact appears in the raw HTML, in the same words users see.
• Each retrieval bot gets a 200 and the full page, not a challenge.
• The rendered DOM adds interactivity, not information. The text diff between raw and rendered is close to empty for primary content.
• No suppression directives appeared during the migration.

Bad

• The raw HTML is an empty shell with a script tag and a loading spinner.
• Headings render, but prices or plan names arrive later from an API call.
• The bot version and the user version disagree on a fact.
• One crawler gets 200 while another gets 403. That is an access problem, not a rendering one, so fix it at the CDN.

A good result means crawlers can now read you. It does not mean an engine will retrieve or cite the page for a given question. That depends on retrieval and generation, which you don't control.

Rendering regressions are quiet. A framework upgrade, a new component library or a redesigned pricing table can move facts back behind JavaScript without anyone noticing.

• After every deploy that touches a priority template: run the checklist for that template.
• After framework, CMS or hosting changes: rerun the full baseline from step 2 on all priority templates.
• After CDN or bot-management changes: rerun the per-crawler status checks. Cloudflare changed default AI crawler settings for some customers on 2026-09-15, so a site can lose access without editing anything.
• Monthly, as of October 2026: spot-check pricing and product pages even if nothing shipped.

Add the raw-HTML grep to your CI pipeline if you can. A failing build is cheaper than discovering months later that a key fact disappeared.

Is prerendering the same as server-side rendering? No. Server-side rendering builds the HTML on each request. Prerendering builds it ahead of time, at build or on first request, and serves the cached result. Static generation builds everything at deploy. All three put text in the first HTML response, which is what AI crawlers need. The choice depends on how often the content changes and what your stack supports, not on which one AI engines prefer.

Do I need SSR if my site only cares about Google? Less so. Googlebot renders JavaScript, so Google's AI Overviews and AI Mode can see client-rendered content once it is indexed. Google's eligibility rule is that a page must be indexed and eligible for a snippet, with no extra technical requirements. The rendering gap matters for ChatGPT, Claude, Perplexity and Meta crawlers, which do not execute JavaScript as of the most recent testing.

Is dynamic rendering for bots considered cloaking? Serving bots prerendered HTML is not cloaking if the content matches what users see. It becomes cloaking when the bot version says something different: extra claims, hidden keywords or facts users can't find. The safest path is serving the same rendered HTML to everyone. If you do serve bots a cached copy, verify the facts match and keep the cache fresh.

Should I add schema while I am migrating templates? It won't help AI citations. Google says structured data isn't required for generative AI search, and Ahrefs' matched-control test found AI Overview citations fell 4.6% after pages added schema. Existing markup does no harm, so there is no need to remove it. Spend the migration effort on getting facts into the raw HTML, which is the lever with evidence behind it.