Tabs, Accordions and Load More: An AI Crawler Check for Interaction-Gated Content

Tabs, accordions, load-more buttons and infinite scroll all hide content until a visitor acts. That is fine for people. For crawlers, what matters is how the content gets onto the page.

There are two patterns, and they behave very differently:

• Hidden by CSS. The server sends every panel's text in the HTML. CSS, a hidden attribute or a native details element keeps it collapsed. A crawler reading the HTML sees all of it.
• Fetched on interaction. The server sends an empty container. JavaScript requests the text only when someone clicks a tab, opens an accordion or scrolls. A crawler that does not run JavaScript never sees it.

Vercel's crawler study found that ChatGPT and Claude crawlers do fetch JavaScript files but do not execute them, so they "can't read client-side rendered content". The evidence we rely on reports this was independently re-confirmed as of June 2026 for GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, PerplexityBot, Meta-ExternalAgent and Bytespider. Googlebot and AppleBot are the exceptions: they render JavaScript.

So the same accordion can be fully visible to Google and empty to ChatGPT search. This is the same mechanism behind lazy-loaded content and AI crawlers, triggered by a click instead of a scroll position.

The practical consequence: if your pricing tiers, specs or FAQ answers live in a fetched panel, those facts may not exist for most AI crawlers. Nothing in your analytics will tell you.

You need a browser, a terminal and about twenty minutes per template.

1. Pick the pages that matter. Start with pricing, product, comparison, FAQ and documentation pages. These carry the facts you most want an AI answer to state correctly.
2. Write down one distinctive sentence from every hidden panel. Open each tab, expand each accordion, click load more once and scroll once. Copy a phrase from each, such as a price, a plan limit or an answer's first line.
3. Get the raw HTML. Run curl -s https://yoursite.com/pricing -o raw.html or use your browser's View Source. Do not use the DevTools Elements panel. It shows the page after JavaScript ran, which is what Googlebot sees, not what OpenAI's crawlers see.
4. Search the raw file for each sentence. Use your editor's find, or grep -c "your phrase" raw.html. Note each one as present or absent.
5. Confirm where absent text comes from. Reload the page with DevTools open on the Network tab, filtered to Fetch/XHR. Click the tab or button. If a request fires and its response contains your sentence, that content is fetched on interaction.
6. Check for embedded data. Some frameworks put content as JSON inside a script tag in the initial HTML. If your sentence appears only there, record it as a gray zone. Vercel notes such data in the initial response may still be indexed, but visible HTML text is the safer form.
7. Repeat the fetch as a crawler. Rerun step 3 with the user-agent string OpenAI documents for OAI-SearchBot, and the strings Anthropic and Perplexity document for theirs. Compare status codes and file contents. A 403 or a challenge page means the firewall, not the markup, is the problem.
8. Look for suppression attributes. Search the raw HTML for data-nosnippet. Templates sometimes apply it to whole panels. Google and Bing both document it as keeping content out of snippets and AI summaries.

If you want to script steps 3, 4 and 7 across many URLs, the patterns in scripts for AI SEO checks you can run yourself are a good starting point.

Every hidden phrase lands in one of a few buckets. Each has a clear next move.

A good page looks boring in this test: every phrase you copied in step 2 appears in raw.html, the crawler fetch returns 200 with the same content, and no panel carries data-nosnippet by accident.

A bad page usually fails in one of two ways. Either a whole tab group is empty in the raw file, or the first screen of items is present and everything behind the button is missing. The second is easy to miss because the page looks fine in View Source at a glance.

The fix is always the same principle: send the content in the first HTML response, then let JavaScript handle the showing and hiding.

Tabs and accordions

Render every panel's text on the server. Collapse panels with CSS, the hidden attribute or the native details and summary elements. Your component library likely supports this with a setting that keeps inactive panels mounted instead of loading them on open.

Load more buttons

Render the first batch in the HTML. Give each further batch its own plain URL (for example /blog?page=2) and link to it with a normal anchor tag. The button can still load results in place for visitors. Crawlers get a path that works without JavaScript.

Infinite scroll

Treat it like load more. Every item a crawler should reach needs to be in some page's raw HTML, reachable by an ordinary link. If an item exists only after a scroll event fires, non-rendering crawlers will not see it.

Key facts

Whatever the component, keep your load-bearing facts (prices, limits, what the product does) in plain HTML text near the top of the page. Stating a specific fact on a fetchable page gives an engine a sentence it can quote.

Be clear about what this buys you. Server rendering decides whether your content exists in the candidate pool at all. It does not make an engine more likely to retrieve or cite you. A critical survey of 45 GEO studies found no technique with a stable, cross-platform causal effect on discoverability. Access is the part you control, so fix it and stop there.

Run this for each template that uses interactive components.

• Listed every tab, accordion, load-more button and infinite scroll region on the template
• Copied one distinctive phrase from each hidden panel and each extra batch
• Fetched raw HTML with curl or View Source, not the Elements panel
• Every phrase found in the raw HTML as visible text, not only inside JSON
• No fetch or XHR request fires on click that returns panel text
• Load more and infinite scroll content reachable through plain paginated URLs with normal links
• Fetch with OAI-SearchBot, Claude-SearchBot and PerplexityBot user agents returns 200 and the same HTML
• No accidental data-nosnippet, nosnippet or max-snippet:0 on content you want quoted
• robots.txt allows the retrieval bots you care about (OAI-SearchBot, not just GPTBot)
• Results recorded with the date and the deploy version tested

To confirm the crawlers are actually requesting these pages, check your logs using the steps in how to see which AI bots crawl your site.

Interaction-gated content breaks silently. A developer swaps an accordion component, and panels that used to ship in the HTML start loading on open. Nobody edits copy, so nobody checks.

Rerun the check:

• After every front-end deploy that touches shared components, templates or the rendering framework.
• After a framework or component library upgrade. Default rendering behavior for inactive tabs can change between versions.
• When you add a new page type that uses tabs, filters or feeds.
• Monthly on your top ten fact-carrying pages, as a spot check that takes a few minutes once scripted.

Run step 7 more often if you sit behind a CDN. Network rules change without anyone touching your site. From 2026-09-15, Cloudflare set Training and Agent crawler categories to blocked by default on ad-displaying pages for new domains and existing free-tier customers. The free AI Crawler Checker on 99visibility.com requests a page as each AI crawler and shows which ones the firewall turns away.

This test answers one question: can a non-rendering crawler read your content. It does not tell you whether that crawler indexed the page, whether an engine retrieved it for a given question, or what the answer said.

It also covers index-time crawlers more cleanly than answer-time fetchers. OpenAI says robots.txt may not apply to ChatGPT-User, which fetches on a user's behalf. The difference is laid out in index-time crawlers vs answer-time fetchers. Content behind a login is a separate problem with a separate answer, covered in whether AI search can see content behind a login or paywall.

To see whether AI engines actually name you once access is clean, you need to ask them. The free checker on this site runs up to three buyer questions and shows whether the engines mention your brand, as a snapshot rather than ongoing tracking.

Does Google see content inside accordions and tabs? Googlebot renders JavaScript, so it can see content that scripts add to the page during rendering. The gap this check finds mostly affects non-Google crawlers, such as OpenAI's, Anthropic's and Perplexity's, which Vercel found do not execute JavaScript. Content in the raw HTML works for both groups, which is why server-rendered panels are the safe default for any page carrying facts you want AI answers to state correctly.

Is text hidden with CSS treated as less important by AI engines? No published evidence we rely on measures that for AI engines. What the evidence does support is simpler: text in the raw HTML can be read by crawlers that do not run JavaScript, and text fetched on click cannot. Focus on the part you can verify. If a collapsed panel's text appears in the server response, the crawler has it. Whether an engine later uses it is outside your control.

Will moving FAQ answers out of accordions get me cited in AI answers? Nobody can promise that. Moving answers into the raw HTML removes a block, so non-rendering crawlers can read them. It does not make retrieval or citation more likely on its own. Adding FAQ schema is not a lever either: Google says structured data is not required for generative AI search, and Ahrefs' matched test found AI Overview citations fell 4.6% after pages added schema.

Is the native details and summary element safe for crawlers? Yes, in the sense that matters for this check. With the native details element, the collapsed text is part of the HTML the server sends, so a crawler reading the raw response sees it. Confirm it on your own pages by running the raw HTML search, since some component libraries wrap the element but still load the inner content with JavaScript on first open.

How do I check infinite scroll pages for AI crawlers? Fetch the raw HTML and count how many items appear. Then scroll in a browser and note an item from the third or fourth batch. If that item is missing from the raw file and only arrives through a network request on scroll, non-rendering crawlers will never reach it. The fix is plain paginated URLs, linked with normal anchors, so every item sits in some page's raw HTML.