Amazonbot: definition, user agent and IndexNow role

Amazonbot is the crawler Amazon runs to request pages from public websites. It is an automated client, not a person in a browser, and it announces itself with the token Amazonbot in its user-agent string.

For a site owner, that token is the handle. You use it to write robots.txt rules aimed at Amazon's crawler, and to spot its requests in your server or CDN logs.

The evidence base for this library does not include Amazon's own crawler documentation. So this entry does not describe which Amazon products use the fetched content. If that matters to your decision, read Amazon's published crawler page before you allow or block it.

Amazonbot works like other crawlers at the access stage. It requests a URL, receives whatever your server returns, and stores or discards it. Three controls decide what it gets.

robots.txt

The Robots Exclusion Protocol (RFC 9309) groups rules by user agent. A group for Amazonbot applies only to Amazonbot:

User-agent: Amazonbot
Disallow: /internal/

If your file has no Amazonbot group, the crawler falls back to the * group. The page on what happens when robots.txt doesn't mention a bot covers that case, and whether bot names are case-sensitive covers spelling.

Firewall and CDN

Bot-management rules sit in front of robots.txt. A WAF can return 403 to a crawler that robots.txt allows. Cloudflare's bot documentation describes how verified and unverified bots are classified and treated differently.

IndexNow

IndexNow is a protocol for telling search engines a URL changed. The participating-engines registry, fetched 2026-08-22, lists seven endpoints: Bing, Yandex, Seznam, Naver, Yep, Internet Archive and Amazonbot. A ping to the protocol can reach Amazonbot as a consumer. The IndexNow documentation says an HTTP 200 response confirms receipt only, with no indexing guarantee.

Amazonbot matters mainly as an access question. You decide whether Amazon's crawler may fetch your pages, and you should make that decision on purpose rather than by accident.

What the evidence does not support is a claim that allowing Amazonbot, or pinging it through IndexNow, gets you into any AI answer. The IndexNow specification never mentions AI, LLMs or chatbots. No study in the evidence base measures an Amazon-side effect.

Rendering is also untested here. Vercel's crawler study and later re-tests found that several AI crawlers (GPTBot, ClaudeBot, PerplexityBot and others) do not execute JavaScript. Amazonbot was not in that tested set. The safe move is the same for every non-rendering crawler: put the facts that matter in the raw HTML. The guide on seeing your site the way an AI crawler does shows how to check.

The next move: confirm that your robots.txt says what you intend for Amazonbot, then confirm your firewall agrees.

• "IndexNow pings make AI cite you faster." The spec never mentions AI, and a 200 only means the ping arrived. Google does not consume IndexNow at all.
• "A request claiming to be Amazonbot is Amazonbot." Any client can copy a user-agent string. Use the steps in verifying a real AI crawler hit before you trust log lines.
• "robots.txt is the whole story." A CDN or WAF rule can turn Amazonbot away silently. The same pattern applies to Bingbot and your firewall.
• "Blocking unknown bots is always safe." It can remove you from systems you wanted to reach. Decide per crawler, with its documented purpose in front of you.

• Applebot: Apple's crawler, one of the few documented to render JavaScript.
• PerplexityBot: Perplexity's search indexing crawler, distinct from its user-triggered fetcher.
• IndexNow: the change-notification protocol whose registry lists Amazonbot.
• robots.txt: the per-agent permission file defined by RFC 9309.

What is the Amazonbot user agent? Amazonbot identifies itself with the token Amazonbot inside its user-agent string. That token is what you match in a robots.txt group and what you search for in logs. The full string can change, so match the token rather than the entire header, and check Amazon's own crawler page for the current form. Remember that a user-agent header alone does not prove the request came from Amazon.

How do I block Amazonbot? Add a robots.txt group with User-agent: Amazonbot and the Disallow paths you want, such as Disallow: / for the whole site. That expresses your preference to a compliant crawler. If you need enforcement rather than a request, add a firewall or CDN rule as well. Test afterwards that the rule hits only Amazonbot and not other crawlers you want to keep.

Does IndexNow send my updates to Amazon? The IndexNow registry, fetched 2026-08-22, lists Amazonbot among seven consuming endpoints alongside Bing, Yandex, Seznam, Naver, Yep and Internet Archive. A submission can therefore be shared with it. What Amazon does with the notification is not documented in the IndexNow spec, and a 200 response confirms only that the ping was received.

Will allowing Amazonbot improve my visibility in AI answers? No published evidence says so. Allowing a crawler can only make your pages eligible to be fetched. Whether any system then retrieves, uses or cites them is outside your control and has not been measured for Amazonbot. Treat the decision as an access and data-use choice, made with Amazon's own documentation, not as a ranking tactic.