SEO Glossary · GEO

PerplexityBot

Blocking every AI crawler in one sweep is the fastest way to disappear from Perplexity answers while gaining nothing on training data. PerplexityBot is the retrieval crawler that feeds citations, not a model-training fetcher, and it behaves differently from Perplexity-User. Treating the two as one is where most 2026 access policies go wrong.

BD
Benoit Demonchaux
Founder · Nautilinks
Updated August 6, 2026 · 10 min read
Key takeaways The essentials in 30 seconds
  • →PerplexityBot and Perplexity-User are two different agents: the first is the indexing crawler you can allow or disallow, the second is a user-triggered fetcher that, per Perplexity's own crawler documentation, generally ignores robots.txt.
  • →A user-agent string is free text and forgeable. Verify PerplexityBot against Perplexity's published IP ranges before you allow-list anything, and before you conclude that a spike of Perplexity hits is real.
  • →robots.txt is a declaration of intent, not an access control. If you actually need enforcement, the rule belongs in the WAF or the server config, and you should test that your CDN is not already blocking AI crawlers by default.
  • →Perplexity's citations are not automatically correct: the 2025 Tow Center study measured a 37 % citation-error rate, the best of the eight tools tested but still a reason to check what page is being credited for which claim.
  • →Crawl eligibility is the cheap half of the work. Being quotable, server-rendered claims, precise numbers, clear attribution, is the half that decides whether the crawl turns into a citation.
  • →Measure Google rankings and Perplexity citations as two separate surfaces. Google's Search Status Dashboard documents no PerplexityBot-specific ranking factor, and conflating the two produces reporting nobody can act on.
3 questions to test your knowledge Read first, the quiz is waiting at the bottom.
Comparison table contrasting PerplexityBot, an indexing crawler bound by robots.txt, and Perplexity-User, a fetcher triggered by a question that generally ignores the file.
Confusing the two agents explains most misconfigured blocks.

A retrieval crawler, not a training scraper

Two agents carry the Perplexity name through your access logs, and treating them as one thing is where most access policies break. PerplexityBot is the indexing crawler: it walks the open web on its own schedule, builds the index Perplexity draws on, and, according to Perplexity's crawler documentation, exists to surface and link websites inside answers rather than to feed foundation-model training. Perplexity-User is something else, a fetcher triggered in real time when someone asks a question that needs a live page. The same documentation states that Perplexity-User generally ignores robots.txt, on the reasoning that the request originates from a human rather than from an automated crawl.

That distinction settles an argument many publishers are still having with themselves. If your objection to AI crawlers is that your prose ends up inside model weights, PerplexityBot is not the target you think it is: blocking it removes you from a citation surface and changes nothing about training. If your objection is commercial, that answer engines keep the content and strip the click, the argument holds, but make it deliberately and price what it costs you.

The volume is now high enough for that price to be real. Perplexity's CEO reported roughly 780 million queries in May 2025, around 26 to 30 million per day, growing more than 20 % month over month at that point. Industry compilations citing Similarweb and DemandSage put Perplexity near 45 million monthly active users by late 2025, though those estimates vary a lot by measurement method and deserve to be read as an order of magnitude, not a number. Either way, it is past the threshold where a serious site should be tracking referrals and citations rather than arguing about whether the surface matters.

Checklist of the classic configuration mistakes made against AI crawlers: generic pattern, wrong layer, delay before the rule takes effect, no cross-check with the source IP.
Five checks that prevent the two most common audit failures.

The user-agent strings, and why you verify them

The documented crawler identity is a single string, and it is worth pinning exactly rather than matching loosely on the word Perplexity:

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)

Perplexity-User announces itself separately, which is what lets you treat the scheduled crawl and the user-triggered fetch as two different decisions: allow the indexer while rate-limiting live fetches, or the reverse if your concern is server load during traffic spikes. That separation is the single most useful thing in Perplexity's crawler documentation, and it is the part competitors covering this topic skip fastest.

A user-agent header is free text. Anyone can send that string from a laptop, and plenty of scrapers do precisely that to inherit whatever allow rule you wrote for the real crawler. Perplexity publishes IP ranges and recommends verifying both the user agent and the source address when you build Cloudflare or AWS WAF rules. Do that. An allow rule keyed on user agent alone is not an allow rule for PerplexityBot, it is an allow rule for anyone who read this article. The same logic applies to the OpenAI crawler and to every other declared AI agent hitting your origin.

robots.txt sets policy, the WAF enforces it

For a site that wants citations, the correct posture is explicit rather than implicit. A blanket allow works, but naming the agent makes your intent auditable six months later when someone asks why an AI bot is in the logs:

User-agent: PerplexityBot
Allow: /
Disallow: /panier/
Disallow: /wp-admin/

User-agent: Perplexity-User
Allow: /

Two operational details matter more than the syntax. Perplexity states that robots.txt changes may take up to 24 hours to propagate, so a fix pushed this morning does not prove anything by lunchtime. And the Perplexity-User block above is close to decorative, since that agent generally ignores the file: it documents your position, it does not enforce it. Anyone treating the AI crawler section of robots.txt as an access control has confused a request with a lock.

This short walkthrough covers the practical check, whether the AI crawlers are actually allowed on a given domain:

Put it into practice?

Nautilinks operates an owned network of editorial media. In-house written articles, transparency disclosures respected, anchor mix calibrated.

See pricing → Buy backlinks service
BD
Benoit Demonchaux Founder · Nautilinks

Founder and operator of Nautilinks. Edits and writes the site's editorial glossary, as well as the content published across the Nautilinks network of editorial media.

Frequently asked questions

Does blocking PerplexityBot hurt my Google rankings?

No. Google's Search Status Dashboard documents no PerplexityBot-related ranking factor, and Googlebot crawls independently of any rule you write for Perplexity. What you lose is presence in Perplexity answers and the referral traffic attached to it. Treat the two as separate surfaces with separate measurements: a robots.txt block scoped to PerplexityBot has no mechanical effect on Google, and inventing a link between them produces reporting nobody can act on.

Why does Perplexity still fetch pages I disallowed in robots.txt?

Almost certainly Perplexity-User rather than PerplexityBot. Perplexity's crawler documentation states that Perplexity-User generally ignores robots.txt because the fetch is triggered by a person asking a question, not by a scheduled crawl. Check the exact user-agent string in your logs. If you need enforcement rather than a declaration of intent, the rule has to sit in your WAF or server config, matched on user agent plus Perplexity's published IP ranges.

Is Perplexity traffic in my logs a security risk?

The declared crawler is not malicious software, it fetches pages like any indexer. The real risk is impersonation: a user-agent header is free text and forgeable. Cloudflare alleged on 4 August 2025 that undeclared crawlers with rotating IPs and Chrome-style user agents were operating after declared ones were blocked, an allegation Perplexity rejected the next day. Verify against the published IP ranges before you trust or allow-list anything.

Should I prioritise Perplexity citations over ChatGPT ones?

Prioritise whichever surface sends you qualified referrals, which you can only know by tracking both. Perplexity is the more citation-native product, it links sources by design, and the widely quoted Semrush analysis of 10 000 answers suggests smaller specialist domains earn citations there more readily than on ChatGPT. That figure is vendor-reported, so use it to justify measuring, not to justify a budget split decided in advance.

How do I allow the indexing crawl but limit live user fetches?

Split the two agents. Allow PerplexityBot in robots.txt for the sections you want indexed, then apply a WAF rate limit keyed on the Perplexity-User agent string combined with the published IP ranges, since robots.txt will not restrain it. That keeps you eligible for citations while capping origin load when a page starts getting pulled repeatedly. Verify with your own logs, not with a checker tool, over a full 24 hours.

My site is allowed and crawled but never cited. What is missing?

Usually extractability or source-worthiness. Fetch the page without executing JavaScript: if the claim you want quoted only appears after client-side rendering, there is nothing to lift. If the HTML is fine, the problem is the writing. Pages that get cited carry specific figures with a named source and a date, unambiguous statements, and clean attribution. Generic prose optimised for keywords gives an answer engine nothing to quote.

Quiz

Test your knowledge

Quiz: PerplexityBot

1/3

What distinguishes Perplexity-User from PerplexityBot in practice?
Newsletter

GEO + SEO analyses and network case studies, in your inbox

Once or twice a month at most. No filler. One-click unsubscribe.

By subscribing you agree to receive our emails. See our privacy policy.

Read next

Related glossary terms

Glossary →

Google AI Mode

Google AI Mode replaces the ten blue links with a cited conversational answer.

AI Overviews

AI Overviews rewrote Google's results page.

Brand mention

Brand mention, explained for working SEOs: why unlinked citations now correlate far more with AI visibility than backlinks, how to track…

ChatGPT SEO

What ChatGPT SEO really means in 2026: Bing indexation, citations, brand mentions and honest measurement.

If you genuinely want to block, the rule goes in the WAF or the server config, matched on user agent and confirmed against the published IP ranges: a Cloudflare custom rule, an AWS WAF string match combined with an IP set, an Nginx map on $http_user_agent returning 403, an Apache mod_rewrite condition. The inverse case is the one that bites more often. Cloudflare announced on 1 July 2025 a default posture of blocking AI crawlers unless creators are compensated, so a site sitting behind a modern CDN may already be blocking PerplexityBot without anyone having decided to. Before you debug your content, check whether your edge is answering 403 to a crawler you assumed was welcome.

Three-step process for confirming a crawler visit: read the declared agent, trace back to the source IP, then conclude on the citation actually obtained.
Policy is read in the documentation, behaviour is read in the server logs.

Reading Perplexity traffic in your logs

Log analysis is where the theory meets what is actually happening. Filter your access logs on the two agent strings, then cross-check the source IPs against Perplexity's published ranges, and you get three buckets: verified indexing crawl, verified user-triggered fetch, and traffic claiming to be Perplexity that is not. That third bucket is not hypothetical. On 4 August 2025, Cloudflare alleged that Perplexity used undeclared crawlers after its declared ones were blocked, reporting rotating IP addresses and ASNs plus user-agent strings impersonating ordinary Chrome on macOS. Perplexity rejected the analysis the following day, on 5 August 2025, calling it technically inadequate.

Pick your side of that dispute if you like, the operational lesson survives either verdict: an apparent PerplexityBot hit is a claim, not a fact, and a request that looks like a Chrome browser may not be one. Sites that care about this instrument their WAF events, not just their analytics, because a browser-shaped scraper is invisible to a tool that filters known bots by name.

A quick way to test what an answer engine can actually reach on your domain:

Where PerplexityBot fits in a netlinking operation

The objective shifts here, and it is worth saying plainly: on this surface the deliverable is being quoted as a source, not accumulating referring domains. Perplexity's product links the sites it cites, so the crawl is the entry ticket and the citation is the outcome. Three things decide whether one turns into the other. Crawl eligibility, which the previous sections cover. Extractability, meaning your claims live in server-rendered semantic HTML rather than behind client-side JavaScript. And source-worthiness, meaning original data, precise figures with attribution, and paragraphs that can be lifted without ambiguity about what they assert.

That last point is where most content programs underinvest, and it is the same discipline that earns a citation inside a generated answer and a link from a journalist. Vague prose is unquotable. A named number with a date and a source is quotable, which is why the pages that get cited in our experience are rarely the ones optimised hardest for keywords.

This one covers the source signals side of the equation:

There is a widely repeated Semrush figure holding that across 10 000 Perplexity answers, 34 % of cited sources came from domains outside the top 1 000 by traffic, versus 18 % for ChatGPT. Treat it as vendor-reported rather than an independent benchmark, but the direction matches what we observe: specialist publishers with narrow, well-sourced pages get cited above their weight class. That is the practical case for working the surface at all, and it is why we ship the same editorial standard across the French media we operate at Nautilinks: a page that reads as a credible source performs on both surfaces, which is also the argument behind the way we approach visibility inside answer engines. If the underlying authority is not there, no crawl policy will manufacture it, and that part still runs on links from real editorial sites, which is what the media catalogue you can browse without an account exists for.

What we see go wrong

The blanket block is the most common. A team reads a headline about AI scraping, drops a wall of Disallow rules covering every agent they can name, and six months later wonders why the brand never appears in answer engines. If you are going to exclude PerplexityBot, exclude it for a stated commercial reason and revisit the decision quarterly.

The mirror mistake is trusting the citation. The 2025 Tow Center and Columbia Journalism Review test of 1 600 queries across eight AI search tools found a 37 % citation-error rate for Perplexity, the lowest of the systems measured, against 67 % for ChatGPT Search and 94 % for Grok-3. Lowest is not low. When a client reports a Perplexity citation, verify that the credited page actually supports the claim, because being cited as the source of something you never wrote is a reputational problem, not a win.

Third: measuring the wrong surface. Google's Search Status Dashboard records the core and spam updates of the period, from the August 2025 Spam Update through the June 2026 Spam Update, and none of them documents a PerplexityBot-specific ranking factor. Google rankings, AI Overview presence and Perplexity citations are three distinct measurements. A Pew Research study from March 2025 found users clicked a traditional result on 8 % of visits when an AI summary was present, against 15 % when it was not, which tells you the click economics of the answer-first surface without telling you anything about your Perplexity performance. Report them separately or the numbers become unusable.

Last, the JavaScript trap. A page that renders its substance client-side can pass every robots.txt check, return 200 to the crawler, and still contain nothing extractable. Fetch your own key pages the way a crawler does, without executing scripts, and read what comes back. If the claim you want cited is not in that response, you are not in the running, whatever your generative engine optimisation checklist says.