Is my page too large for Google to read in full?

Googlebot has a hard ceiling on how much of a page it reads. Gary Illyes set it out in Google's Search Central blog post of March 31, 2026, "Inside Googlebot: demystifying crawling, fetching, and the bytes we process":

• Googlebot currently fetches up to 2MB for any individual URL, excluding PDFs.
• For PDF files, the limit is 64MB.
• Bytes after the 2MB threshold are "entirely ignored. They aren't fetched, they aren't rendered, and they aren't indexed."
• For other crawlers that don't specify a limit, the default is 15MB.

The same post puts the risk in proportion. Google writes that for the vast majority of the web, a 2MB HTML payload is massive, and you will never hit this limit. A typical marketing page, blog post or pricing page sits far below it.

So the reality is simple. The cap exists, it cuts content off silently, and it almost never applies. The next move is to check only the pages that could plausibly be huge. Our definition of the crawl payload limit covers the term itself.

The counter is per URL, not per site. Google's post says sub-resources have separate counters. An external JavaScript bundle, a stylesheet or an image file is fetched as its own URL and does not eat into the HTML page's budget.

What does count is everything inside the HTML document itself. Pages that approach the cap usually do it by inlining things that could live elsewhere:

• Images embedded as base64 strings instead of linked files.
• Large inline SVG graphics repeated across the page.
• Big blocks of inline CSS or JavaScript.
• Framework state serialized into the HTML so the page can hydrate in the browser.
• Very long listings, such as thousands of products or comments rendered on one URL.

The order matters. Googlebot reads from the top. If your main content sits below a heavy block of inlined code, that content is the part most at risk of falling past 2MB.

Google's AI features don't have their own crawler. They draw on the same Googlebot crawl and the same index, which is why there is no separate robots.txt control for AI Overviews.

Google's AI features documentation states that to be eligible as a supporting link in AI Overviews or AI Mode, a page must be indexed and eligible to be shown in Google Search with a snippet. It adds that there are no additional technical requirements. Text beyond the 2MB point isn't indexed, so it can't count toward that eligibility. A fact that only appears past the cutoff is, as far as Google is concerned, not on the page.

Non-Google AI crawlers are a different case. Google's 2MB figure describes Googlebot only and is not a ceiling on ChatGPT, Claude or Perplexity crawlers. For those crawlers, a more common way to lose content is JavaScript. Vercel's crawler study found that ChatGPT and Claude crawlers fetch JavaScript files but don't execute them, so they can't read client-side rendered content. If you are asking whether AI reads your whole page, rendering is usually the better first question than size.

PDFs

PDFs get a 64MB allowance from Googlebot, far more than HTML. A long PDF report is rarely cut off by size. Whether AI engines use PDF content well is a separate question, covered in can AI search engines read PDFs.

Smaller is not better past the threshold

The cap is a cliff, not a slope. Nothing in Google's post suggests that a 300KB page beats an 800KB page. Trimming a page that is already under 2MB changes nothing about how Googlebot reads it. Don't sell or buy page-size reduction as an AI visibility tactic.

The number can change

Google wrote that Googlebot "currently" fetches up to 2MB. The figures here are as of October 2026. Recheck Google's documentation before you build a process around a specific number.

Size is not the only cutoff

A page can be fully fetched and still be kept out of AI answers by a stray noindex, nosnippet or max-snippet:0. Those controls fail more often than the byte cap does, and they are easier to check.

You only need to do this once for your heaviest templates, then again after a major redesign.

1. Pick the candidates. Look for long category pages, single-page apps that inline their state, and pages with embedded images or very long tables.
2. Measure the raw HTML. Request the page without running JavaScript and record the bytes returned, including headers. The free page size check does this, or use curl:

curl -s -o /dev/null -w '%{size_header} %{size_download}\n' https://example.com/page

3. Compare against the thresholds. Flag anything over 2MB for Googlebot, or over 15MB for crawlers that set no limit. Everything under is fine on size.
4. If a page is over, fix the cause. Move base64 images and inline SVG to linked files, move inline scripts and styles to external files, paginate very long listings, and put the main content high in the HTML.
5. Check rendering while you are there. Compare raw HTML with the rendered page using the free JavaScript rendering check. Content that appears only after JavaScript runs is a far more common gap for non-Google AI crawlers.

The crawler payload limit playbook walks through the same steps at site scale. What you control here is whether your content sits inside the bytes Googlebot reads. Whether Google then uses it in an answer is not in your control.