Why do AI crawlers download JavaScript files they don't run?

Downloading a file and executing it are two different jobs. A crawler can request app.js, store the bytes, and never run a line of it. That's what Vercel observed.

In its study of AI crawler traffic, Vercel found that ChatGPT's crawlers fetched JavaScript files on 11.50% of requests and Claude's on 23.84%. Vercel's conclusion was blunt: "they don't execute them. They can't read client-side rendered content." You can read the full breakdown in our summary of Vercel's AI crawler study.

Why fetch a file you won't run? Neither OpenAI nor Anthropic documents a reason. One plausible reading is that a crawler follows the URLs it finds in a page, and a <script src> tag is a URL. That is an inference, not a published fact. What matters for you is the outcome, and the outcome is documented: the content your scripts would have produced never reaches the crawler.

So a page that builds its pricing table, product copy or FAQ in the browser can look complete to a visitor and nearly empty to an AI crawler. The next move is to check what your HTML contains before JavaScript rendering happens.

Server logs record requests. They don't record what the requester did with the response.

When you see GPTBot or ClaudeBot pulling .js files, you know three things:

• The crawler reached your server (your firewall didn't block that request).
• It received a status code, which you can read in the log line.
• It downloaded the file.

You don't know whether anything was executed, rendered or indexed. Based on the Vercel evidence, assume it wasn't.

This is why JS fetches are a misleading comfort. A busy log full of AI crawler hits on your bundles can sit alongside zero useful content captured. It's one common reason teams see AI crawlers visit but AI never cites the site.

Use logs for what they're good at: confirming access and status codes. Our playbook on reading server logs for AI crawler status codes covers that. Don't use them to judge whether your content was understood.

The split is consistent, and it matters which engine you care about.

Vercel's original data covered ChatGPT and Claude. Independent re-testing as of June 2026 found the same result for every non-Google, non-Apple crawler in the table.

What this means: client-side rendering is mostly a non-issue for Google AI Overviews and AI Mode, because Googlebot renders pages (Google's JavaScript SEO documentation describes this). It's a real issue for ChatGPT search, Claude and Perplexity. If those engines matter to you, you can't lean on Google's rendering to cover them.

Note that OpenAI runs separate crawlers for training and search. Allowing one doesn't cover the other, and none of them render scripts.

This is the strongest finding in the whole GEO evidence base, and it's still worth knowing its edges.

• Sample. Vercel's data is a one-month snapshot from December 2024, mostly on nextjs.org, a site skewed toward one framework.
• Coverage. Microsoft Copilot was excluded because it lacked a unique user agent.
• Incentive. Vercel sells server-side rendering hosting. The finding has since been re-confirmed independently, which matters more than the incentive.
• Data in the initial HTML. Vercel notes that JSON or React Server Component payloads included in the initial HTML response may still be indexed. Content fetched later by a script is the problem.

There's also a boundary on what fixing this buys you. Getting content into the raw HTML decides whether it exists for the crawler at all. It doesn't make an engine more likely to pick or cite you. That depends on retrieval and generation, which you don't control. Think of it as removing a block, not adding a boost.

Run a simple comparison on your most important pages: pricing, product, about, and any page that states a fact you'd want an AI answer to repeat.

1. Fetch the raw HTML with no JavaScript. From a terminal:

curl -s https://example.com/pricing | grep -i "per month"

If your price, plan names or key claims don't appear, a non-rendering crawler can't see them either.

2. Compare with the rendered page. Open the same URL in a browser and list what's visible. Anything present only after rendering is at risk. Our scripts for AI SEO checks include a fuller version of this diff.
3. Check content behind interaction. Tabs, accordions and "load more" buttons that pull content via a script fail the same way.
4. Check meta robots too. A noindex or nosnippet tag injected by script raises its own questions, covered in whether AI crawlers see robots meta tags added by JavaScript.
5. Fix at the source. Server-side rendering, static generation or prerendering for the load-bearing content. Our guide to JavaScript rendering and AI crawlers compares the options.

Raw HTML only helps if the crawler can reach the page. A robots.txt that allows a bot doesn't stop a firewall from returning 403. The AI Crawler Checker requests a page as each AI crawler and shows which ones get turned away.

Does GPTBot render JavaScript? No. Vercel found ChatGPT's crawlers fetched JavaScript files on 11.50% of requests but didn't execute them, and independent re-testing as of June 2026 found the same for GPTBot, OAI-SearchBot and ChatGPT-User. Content that only appears after scripts run should be treated as invisible to OpenAI's crawlers. Put anything you want ChatGPT search to read into the server-delivered HTML.

Should I block AI crawlers from my JavaScript files? There's no evidence it helps or hurts. Blocking JS files won't make crawlers read more of your content, and allowing them won't make crawlers render your page. The decision that matters is whether your key content is in the raw HTML. Leave script access as it is unless bandwidth from crawler JS fetches is a real cost for you.

Does client-side rendering hurt my visibility in Google AI Overviews? Much less than elsewhere, because Googlebot renders JavaScript. Google's AI Overviews and AI Mode draw on pages that are indexed and eligible for a snippet in Google Search. The rendering gap mainly affects ChatGPT, Claude, Perplexity and Meta's crawlers. If those engines matter to you, server-rendering the important content is still worth doing.

If my content is in the raw HTML, will AI engines cite me? Not necessarily. Raw HTML removes one block: the crawler can now read the content. Whether an engine retrieves your page for a question, and whether it cites you, depends on retrieval and generation steps nobody outside the operator controls. What you can do is make sure clear, specific facts are fetchable, so there's a sentence available to quote.

Why do Claude's crawlers fetch more JavaScript than ChatGPT's? Vercel reported the numbers (23.84% of Claude crawler requests versus 11.50% for ChatGPT) but neither Vercel nor Anthropic explains the difference. Treat it as an observed pattern from one December 2024 snapshot, not a documented behaviour. The practical point is the same for both: neither executed the files they downloaded.