PerplexityBot: definition and how it works

99Visibility Research · Updated October 21, 2026 · 5 min read

Short answer

PerplexityBot is the crawler Perplexity uses to build its search index, and Perplexity says it is not used to crawl content for AI foundation models. Allowing it in robots.txt, and letting it through your firewall, makes your pages eligible for Perplexity's index. Eligibility is not a promise: it does not guarantee retrieval or a citation for any query.

Key takeaways

PerplexityBot crawls for Perplexity's search index, and Perplexity's documentation says it is not used to train foundation models.
Blocking PerplexityBot in robots.txt removes your pages from the index Perplexity answers from, so check that no blanket AI rule catches it.
A CDN or firewall can turn PerplexityBot away even when robots.txt allows it, so test both gates.
PerplexityBot does not execute JavaScript, so content that appears only after scripts run is likely invisible to it.
PerplexityBot is separate from Perplexity-User, the user-triggered fetcher that Perplexity says generally ignores robots.txt.

What PerplexityBot is

PerplexityBot is Perplexity's search indexing crawler. It fetches public pages and stores them so Perplexity can retrieve them when it answers a question. Perplexity's crawler documentation states the bot is used for search indexing and is "not used to crawl content for AI foundation models."

That makes it an access-stage bot. It decides whether your page can exist in Perplexity's index at all. It does not decide whether your page gets picked for a given answer.

This entry is a short definition. For the full comparison with Perplexity's other agent, including firewall setup, read PerplexityBot vs Perplexity-User: robots.txt and WAFs. For a line-by-line reading of the vendor docs, see Perplexity's crawler documentation, digested.

How it works

PerplexityBot identifies itself with its own user-agent token, so robots.txt rules can target it by name. Perplexity's documentation lists the exact user-agent string. Copy it from there rather than from a blog post, because strings change.

A rule that allows it looks like this:

User-agent: PerplexityBot
Allow: /

Three things shape what the bot actually sees:

robots.txt. Rules are per bot. A Disallow for PerplexityBot tells it to stay out. If your file has no group naming it, the Robots Exclusion Protocol (RFC 9309) says it follows your User-agent: * group. More on that in whether an unlisted AI bot is allowed.
Your network edge. A firewall, CDN or bot-management rule can return a 403 or a challenge page before robots.txt is ever consulted.
Rendering. Vercel's crawler study found ChatGPT and Claude crawlers fetch JavaScript files but do not execute them, and independent testing in June 2026 found the same for PerplexityBot. Prices, product names and key claims that only appear after hydration may never reach the index.

How Perplexity then uses its index at answer time is a separate question, covered in does Perplexity have its own search index.

Why it matters for AI visibility

If PerplexityBot can't fetch your pages, Perplexity's index has nothing of yours to retrieve. That is the one part of Perplexity visibility you fully control.

The most common real-world error is a blanket AI block. Someone adds a rule meant to stop training crawlers, and it also removes the search crawlers that put you in AI answers. PerplexityBot is a search crawler, so blocking it to opt out of training does not match what Perplexity says the bot is for.

The second error is invisible from outside. OpenAI's help center states that ChatGPT search eligibility needs both a robots.txt allow and a host or CDN that lets the crawler's traffic through. The same two-part logic applies to any crawler, including PerplexityBot. Cloudflare also changed its AI crawler defaults on 2026-09-15, blocking Training and Agent categories by default on ad-displaying pages for some accounts while leaving Search allowed. Check your own settings rather than assuming.

The next move: request a page as PerplexityBot and see what comes back. The AI Crawler Checker does this for each AI crawler and shows which ones your firewall turns away. To confirm a hit in your logs is genuine, use this free way to verify a real AI crawler hit.

Common misconceptions

"Allowing PerplexityBot gets me cited." It buys eligibility only. Retrieval is decided per query by Perplexity, and a July 2026 survey of 45 GEO studies found no technique with a stable, cross-platform causal effect on discoverability.
"Blocking PerplexityBot stops Perplexity from ever fetching my site." Perplexity-User handles user-triggered fetches, and Perplexity says it generally ignores robots.txt rules. Index-time and answer-time exclusion are different controls.
"robots.txt is the whole story." A WAF or CDN can block a crawler robots.txt allows. Test reachability directly.
"It works like Googlebot." Googlebot renders JavaScript. PerplexityBot, per June 2026 testing, does not. Server-render the content that matters.
"Rules for one AI bot cover the others." Each vendor's tokens are independent. The same split exists at OpenAI, explained in OpenAI crawlers, robots.txt and firewalls.

Frequently asked questions

Does PerplexityBot train Perplexity's AI models?

Perplexity's crawler documentation says PerplexityBot is for search indexing and is not used to crawl content for AI foundation models. That is Perplexity's own statement, and no outside party can audit it. If your goal is to stay out of training while staying in Perplexity's search results, blocking PerplexityBot works against the second goal without addressing the first.

What is the PerplexityBot user agent?

PerplexityBot sends a user-agent string containing the token PerplexityBot, which is what robots.txt rules match on. Take the full current string from Perplexity's crawler documentation, since vendors update them. A user-agent can be spoofed, so confirm suspicious hits with a verification method such as reverse DNS or the vendor's published details before trusting your logs.

If I allow PerplexityBot, will Perplexity cite my site?

Not necessarily. Allowing it makes your pages eligible for Perplexity's index. Whether a page is retrieved and cited for a given question is decided by Perplexity at answer time, and no published study shows a reliable way to control that. What you can control is access: robots.txt, firewall rules and whether key content is in the raw HTML.

Why would PerplexityBot be blocked if my robots.txt allows it?

Your CDN, firewall or bot-management layer can reject the request before robots.txt matters, often with a 403 or a challenge page. Nothing in robots.txt reveals this. Request a page using the PerplexityBot user agent and check the status code, then review your CDN's bot settings, especially if your provider changed AI crawler defaults recently.

Does PerplexityBot read JavaScript-rendered content?

Independent testing in June 2026 found PerplexityBot does not execute JavaScript, consistent with Vercel's earlier finding for ChatGPT and Claude crawlers. Content injected after page load, such as prices or product details from client-side code, may not be indexed. Compare your raw HTML with the rendered page and move anything load-bearing into the server response.

Sources

This page is AI-assisted. It was written from a verified evidence base and checked against our editorial policy. Engine behaviour changes often; facts are as of October 21, 2026.