Ecommerce AI Search Product Pages: Check That Crawlers Can Read Your Product Data

Vercel's crawler study (published December 2024, updated February 2025) found that ChatGPT and Claude crawlers fetch JavaScript files but do not run them. JavaScript made up 11.50% of ChatGPT's crawler requests and 23.84% of Claude's, yet neither executed the code. Independent retesting in June 2026 found the same for OAI-SearchBot, Claude-SearchBot and PerplexityBot.

What that means for a shop: if a script writes the price into the page after load, those crawlers never see a price. The same goes for stock status, sizes and specs.

Googlebot is the exception. It renders JavaScript, so Google AI Overviews and AI Mode are less exposed. ChatGPT search, Claude and Perplexity depend on their own crawlers, and those read the HTML as the server sends it.

This is an access problem, the first of four stages. It decides whether your facts exist for an engine at all. It does not decide whether the engine retrieves your page for a query or cites it. Those later stages belong to the engine. The raw HTML is the part you control, so fix it first. For the wider picture of how shops appear in answers, see how an online shop shows up in AI shopping answers.

Work by template, not by product. One broken template breaks every product that uses it.

1. Pick one URL per template: a simple product, a product with variants, an out-of-stock product, a bundle, and a category page.
2. Fetch the raw HTML without running JavaScript. In a browser, use View Source, not Inspect (Inspect shows the rendered page). From a terminal:

curl -s https://www.example.com/products/oak-desk -o raw.html
grep -c "249.00" raw.html

3. Search the raw file for each fact as visible text: the price with its currency, the availability wording, and the three to five specs a buyer compares.
4. Open the page in a browser and switch every variant. Note which values change. Then search the raw file for each variant's price and stock.
5. Render the same URL in a headless browser and compare the text, or use the free JavaScript rendering check. Anything present only after rendering is invisible to most AI crawlers.
6. Check access per crawler. Read robots.txt for OAI-SearchBot, PerplexityBot, Claude-SearchBot and Googlebot. Then request the page as each crawler, because a firewall can block a bot robots.txt allows. The free AI crawler checker does this, as does 99Visibility's AI Crawler Checker.
7. Look for suppression tags on the page and the price block: noindex, nosnippet, max-snippet:0 and data-nosnippet. Here is what those tags do to Google AI Overviews.
8. Record a pass or fail per fact, per template. That record is your baseline for the next run.

Run each template against this list. Every item should be true in the raw HTML, before any script runs.

• The product name appears as visible text, not only in an image.
• The current price appears with its currency symbol or code.
• The default variant's price matches what a shopper sees on load.
• Every variant's price is present (for example, a size and price table), not fetched on click.
• Availability is stated in words ("In stock", "Ships in 3 days", "Out of stock").
• Key specs (dimensions, materials, capacity, compatibility) are in the HTML, not in a tab that loads on click.
• Shipping cost or threshold, if it is a buying factor, is stated on the page.
• No cookie or consent wall replaces the content in the HTML. See whether a cookie banner can hide your site from AI.
• The page is not noindex, and no nosnippet or data-nosnippet covers the price or specs by accident.
• robots.txt allows the search crawlers you want, per bot.
• Each crawler gets a 200 response, not a 403, a challenge page or a CAPTCHA.
• The raw HTML is under the size crawlers fetch. Check your page size if templates inline large scripts.

A good result is boring. The raw file and the rendered page say the same thing about price, stock and specs.

A bad result names its fix. A missing price means the template needs server-side rendering or prerendering for that block. A 403 means a firewall or CDN rule needs an exception for verified crawlers. An inherited data-nosnippet means someone hid a widget and caught the price with it.

One caution on markup. A price that exists only inside JSON-LD is not the same as a visible price. Google says AI Overviews and AI Mode have no additional technical requirements beyond indexing and snippet eligibility, so treat markup as optional, and keep it consistent with the visible price.

Most failures come from a few patterns. Check these first.

Variant selectors

The server renders the default variant. Picking a size or colour triggers a request that swaps in a new price and stock state. The crawler only sees the default. The fix is to render all variant prices and stock in the initial HTML, then let the selector highlight one.

Tabs and accordions

Collapsed content is fine if the text is already in the HTML. It fails when opening the tab fetches the text. Ask your developer which one your theme does.

Price and currency scripts

Geolocation, currency conversion and personalised pricing scripts often blank the price until they run. Render a default price server-side.

Apps and widgets

Stock counters, "back in stock" apps and review widgets usually inject content with JavaScript. On hosted platforms, theme apps are a common cause. The Shopify AI SEO guide covers platform specifics.

Access layers

A bot-management rule can turn away a crawler that robots.txt allows. OpenAI's crawler documentation is clear that robots.txt settings are per bot: OAI-SearchBot controls ChatGPT search, while GPTBot governs training only. Blocking one does not block the other.

A 252,000-trial factorial study on competitive GEO found that "including explicit price information and a recent timestamp also helps consistently." That is the closest thing in the evidence to a positive signal for product content.

Read the setup before you act on it. The study used a synthetic corpus of 100 ecommerce blogs. Each trial compared two candidate documents already placed in the model's context. No search engine was called. The authors work for a company that ships a GEO checklist. So the finding describes what happens once a page is already in front of the model. It says nothing about whether a price gets your page retrieved in a live engine.

The honest takeaway is modest. A plainly stated price is a specific fact an engine can quote, if it retrieves your page. That is a reason to put prices in visible text. It is not a promise of citations, rankings or shopping placements, and nobody can make that promise. A critical survey of 45 GEO studies (July 2026) found no technique with a stable, cross-platform causal effect on discoverability.

Product data breaks without anyone touching a product. Tie the check to the changes that cause breakage.

• Every theme or template deploy. Re-run the raw fetch on one URL per template.
• Every new app or script that touches price, stock or variants. These are the most common cause of script-only prices.
• Every firewall, CDN or bot-management change. Cloudflare set Training and Agent crawlers to blocked by default on ad-displaying pages for new domains and existing free-tier customers from 15 September 2026, while Search stayed allowed. Defaults like that can change access without an edit on your side.
• Monthly. Re-run the crawler access check on your top templates.
• Quarterly. Run the full checklist across every template, including category pages.

Once the data is readable, the next question is whether engines name your products. Here is how tracking works for individual products and SKUs.