Find Key Facts That Only Appear After JavaScript

When a crawler requests a page, your server returns an HTML document. A browser then runs the JavaScript, which can add, replace or remove text. Sites built with client-side rendering often send a near-empty shell and fill it in with scripts.

That second step is where AI crawlers drop out. Vercel's crawler study (December 2024, updated February 2025) found that ChatGPT and Claude crawlers do fetch JavaScript files (11.50% and 23.84% of their requests) but don't execute them, so they can't read client-side rendered content. The evidence base behind this library records that the finding was independently re-confirmed in June 2026 for GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, PerplexityBot, Meta-ExternalAgent and Bytespider. You can read the detail in our summary of Vercel's AI crawler study.

Googlebot is the exception. Google documents that it renders JavaScript, so Google AI Overviews and AI Mode are not affected the same way. This creates a trap: a page that indexes cleanly in Google can still be blank to every other AI engine.

What it means for you: the raw HTML is the only copy of your page that most AI crawlers ever see. If a fact is not in it, those crawlers cannot quote it, check it or correct a wrong version of it.

Not every sentence needs to be in the raw HTML. The facts below are the ones buyers ask AI tools about, and the ones AI answers most often get wrong. Stating a specific fact on a fetchable page gives an engine a sentence it can quote.

Work through this list for each key page:

• Prices. Every plan price, currency and billing period (monthly, annual, per seat).
• Plan names and limits. Tier names and what each includes, such as seats, usage caps or included features.
• Product and service names. The exact names you want used, plus a one-line description of each.
• Company identity. Legal or trading name, what the company does in one sentence, founding year, headquarters location.
• Contact details. Phone number, email, physical address, opening hours for local businesses.
• Locations and service areas. Cities, regions or countries you serve.
• Integrations and compatibility. Platforms you connect to, if buyers choose you on that basis.
• Certifications and compliance. Any certification you would want named in a comparison answer.
• Policies that affect buying. Free trial length, refund terms, minimum contract.
• Page titles, headings and meta tags. The title, H1 and meta description, plus any robots meta tags.

The last item matters more than it looks. A robots meta tag added by script may not be seen at all, which is covered in whether AI crawlers see robots meta tags added by JavaScript.

This takes about ten minutes per page. Start with your pricing page, homepage, main product pages and contact page.

1. Write down the exact strings. For each fact on the checklist, note the exact text as it appears on the live page, for example the price "$49" and the plan name "Starter".
2. Fetch the raw HTML. In a browser, open view-source on the page (not the Elements panel in developer tools, which shows the rendered DOM). Or fetch it from the command line:

curl -s https://example.com/pricing > raw.html

3. Search for each string. Use find in the source view, or grep the saved file:

grep -c "Starter" raw.html

A count of zero means the string is not in the HTML the server sent.

4. Check where a match sits. If the string appears only inside a script tag, it may be a JSON payload rather than visible text. Vercel notes that JSON or React Server Components data in the initial HTML response may still be indexed, but visible text in the body is the safer state. Record it as "in a script only".
5. Repeat with JavaScript disabled in the browser. Turn off JavaScript in browser settings and reload. Anything that disappears is client-rendered. This also catches text that the server sends but scripts later overwrite.
6. Compare against the rendered page. For a full diff of raw HTML against the rendered DOM, follow the raw HTML vs rendered DOM check.
7. Confirm crawlers actually get the HTML. Your server may send different responses to bots, or a firewall may block them before any HTML is served. The AI Crawler Checker requests a page as each AI crawler and shows which ones get turned away.
8. Log the result per fact. A simple sheet works: page, fact, present in body, present in script only, missing.

Expect mixed results. Many sites render the header and footer on the server and load the useful middle of the page with scripts.

The conflict case deserves attention. A server-rendered placeholder or cached default price can be the only version an AI crawler sees. That is one plausible route to an AI answer stating the wrong price.

When a fact fails the check, the cause is usually one of a few patterns:

• Pricing widgets. Toggles between monthly and annual billing that fetch prices from an API after load.
• Tabs, accordions and "load more". Content fetched only when a visitor clicks.
• Headless CMS front ends. The CMS stores the content, but the front end may fetch it in the browser. See whether a headless CMS makes a site invisible to AI crawlers.
• Location and contact blocks. Store finders, maps and click-to-reveal phone numbers.
• Personalised or geolocated content. Prices or currency set by script after detecting the visitor's region.
• Content behind a login or paywall. No rendering setting fixes this, because crawlers do not log in.

Knowing the pattern tells you which team owns the fix: the frontend team for widgets and tabs, the CMS owner for headless setups, the growth team for personalisation.

The fix is to put the fact into the HTML the server sends. The usual routes are server-side rendering, static generation, or prerendering for the pages that matter. The SSR and prerendering rollout plan covers sequencing and testing.

For small gaps, a lighter fix often works. Render the default price and plan names in the HTML, and let scripts handle the toggle. Put contact details in plain text in the footer rather than behind a reveal.

Be clear about what this buys. Fixing rendering is hygiene, not optimisation. It decides whether your facts exist for the crawler at all. It does not make an engine more likely to retrieve or cite your page, and nobody outside the engine controls that choice. The honest answer to whether SSR increases AI citations is that it removes a block. No published before-and-after study has measured the effect on citations.

Avoid one shortcut. Serving AI crawlers different content from what visitors see is cloaking. It is a risk with every engine and never a fix.

Rendering breaks quietly. Nobody changes a setting called "hide from AI". A component gets refactored and the price moves into a client-side fetch.

A practical rhythm:

• After every relevant deploy. Any change to templates, pricing components, the CMS or the framework.
• After price or plan changes. Confirm the new numbers are in the raw HTML, and that no old values remain in placeholders.
• Monthly spot-check. Run the steps on your pricing, homepage and contact pages.
• Before and after a migration. Framework upgrades and redesigns are the most common points of regression.

If you can, automate it. A scheduled job that fetches key pages and greps for each checklist string catches regressions the day they ship.