JavaScript Rendering and AI Crawlers: What They See and What They Miss

A crawler that requests your page keeps the HTML your server sends. On a client-rendered site, that HTML is often a near-empty shell: a root element, some script tags and very little text. The browser builds the visible page by running JavaScript. A crawler that skips that step stores the shell.

Vercel measured this on its own network. Its study, published in December 2024 and updated in February 2025, found that ChatGPT and Claude crawlers do fetch JavaScript files (on 11.50% and 23.84% of their requests) but "they don't execute them. They can't read client-side rendered content." Our breakdown of the Vercel AI crawler study covers the method in detail.

Read the finding with its limits in view:

• It was a one-month snapshot on nextjs.org, a sample skewed toward Next.js traffic.
• Microsoft Copilot was excluded because it has no unique user agent.
• Vercel sells hosting built around server-side rendering, so it has a commercial interest in the result.

The result has since been re-tested independently. As of June 2026, none of the AI crawlers in that retest rendered JavaScript.

What it means for you: if your pricing, product names or positioning load through JavaScript, those crawlers store a page without them. No engine can quote a sentence its crawler never received.

The split is clean, and it is the reason a site can look fine in Google and empty to ChatGPT.

Google is the exception. Googlebot renders JavaScript, and Google's rule for AI Overviews and AI Mode is that a page must be indexed and eligible to show in Search with a snippet, with "no additional technical requirements." A client-rendered page can still reach Google's AI features, which we cover in whether AI Overviews need server-side rendering.

The other eight rows are the reason to fix it anyway. Those crawlers feed ChatGPT search, Claude, Perplexity and Meta AI, and none of them sees what your scripts build.

Rendering problems rarely cover a whole site evenly. They cluster in a few patterns.

• Full client-side rendering. Single-page apps that ship an empty shell and build everything in the browser. See the definition and risk of client-side rendering.
• Data fetched after load. Pricing tables pulled from an API, plan comparisons, review widgets and feature lists that arrive after the first response.
• Content behind interaction. Accordions, tabs, "load more" buttons and infinite scroll that inject text with a fetch when someone clicks. If the text is already in the HTML and only hidden visually, a non-rendering crawler still receives it. If it is fetched on click, it does not.
• Head tags set by JavaScript. A title, canonical or robots directive written by a script is missing for any crawler that does not render. That cuts both ways, as our note on robots meta tags added by JavaScript explains.

One nuance from Vercel: JSON or React Server Component payloads included in the initial HTML response may still be indexed. The risk is content that streams in later or loads in a separate request.

The test is simple: compare what a non-rendering crawler receives with what a person sees. The gap is your exposure.

1. Pick your load-bearing pages. Start with five to ten: home, pricing, top product or service pages, about, and any comparison pages.
2. Fetch the raw HTML. Request each URL with JavaScript off, for example curl -s or your browser's view-source. Save the response.
3. Render the same URL. Load it in a headless browser, wait for scripts to settle, and save the final DOM.
4. Extract visible text from both. Strip tags, scripts and styles so you compare words, not markup.
5. Diff the two. List every line that appears only in the rendered version.
6. Check the list against your key facts. Prices, plan names, product names, headings, your one-line description of what you do, and named entities. Note what share of the primary content exists only after JavaScript runs.
7. Repeat for interactive states. Open tabs and accordions in the rendered version and check whether that text was in the raw HTML.

For a quick manual pass, disable JavaScript in your browser and reload the page. Whatever disappears is a candidate problem. The full procedure is in our raw HTML vs rendered DOM check.

One caution before you trust the raw fetch: a firewall or CDN can answer an AI crawler's user agent with a 403 even when your browser gets a 200. The AI Crawler Checker requests a page as each AI crawler and shows which ones get turned away.

It helps to separate the four stages an AI answer passes through.

• Parametric. What a model knows with no web access, frozen at its training cutoff. Nothing you ship this quarter reaches it.
• Access. Whether a crawler can fetch and store your page, and what text it gets. Rendering lives here, and you control it.
• Retrieval. Whether your page enters the model's context for a given query. The engine decides.
• Generation. Which retrieved pages the model uses and cites. The model decides.

Server-side rendering is access hygiene. It decides whether your content exists in the candidate pool at all. It does not make an engine more likely to pick you over another page. No published before-and-after study yet measures whether shipping SSR changes citations, which is why we answer whether SSR will increase your AI citations with a plain "it removes a block, nothing more is proven."

The wider evidence agrees. A July 2026 critical survey of 45 GEO studies (arXiv:2607.14035) found no technique with a stable, cross-platform causal effect on discoverability. What does hold up is narrower: stating a specific fact, such as a price or a one-line definition, in HTML a crawler can fetch gives the engine a sentence it can quote.

Fix things in the order a crawler meets them.

1. Confirm access. Check robots.txt per bot. ChatGPT search uses OAI-SearchBot, and GPTBot governs training only, per OpenAI's crawler page. Then confirm your firewall or CDN lets those crawlers through.
2. Run the diff on your load-bearing pages and write down which facts are missing from the raw HTML.
3. Move those facts into the server response. Server-side rendering, static generation and prerendering all work, as long as the first HTML response contains the text. Start with the routes that carry pricing and product facts. Our SSR and prerendering rollout plan covers sequencing.
4. Move head tags server-side. Titles, canonicals and robots directives should be in the HTML the server sends.
5. Re-run the diff after deploy and log the date, so you can line up any later change in AI answers against a known fix.
6. Add the diff to release checks. A new pricing widget can quietly move your most important numbers back into JavaScript.

• Testing in Google and assuming the rest match. Googlebot renders. The ChatGPT, Claude and Perplexity crawlers tested as of June 2026 did not. A page that looks complete in Google's tools can be empty to them.
• Treating SSR as a citation guarantee. It makes your content readable. Retrieval and selection stay with the engine.
• Reaching for schema instead. Google says structured data isn't required for generative AI search, and Ahrefs' matched test found AI Overview citations fell 4.6% on pages after schema was added. JSON-LD injected by JavaScript is also invisible to non-rendering crawlers. The schema vs server-side rendering comparison sets out why rendering is the access lever and schema is not.
• Checking only the homepage. Pricing and product pages are where client-side data fetching usually lives.
• Serving crawlers different content from people. Prerendering the same content for bots is fine. Showing AI crawlers text that users never see is cloaking, and hidden or injected text aimed at AI systems is a manipulation risk, not a shortcut.
• Fixing rendering behind a block. If your CDN returns 403 to OAI-SearchBot, perfect HTML still never arrives.