The short answer.
To be cited by Perplexity, a page has to clear five steps in order: it is crawled, it is indexed, a passage from it is retrieved for the question, that passage survives ranking, and the answer links to it. The first and fourth steps are where most of the work is, and their fixes are very different.
Perplexity describes itself as an answer engine, one that works by “searching the web, identifying trusted sources, and synthesizing information into clear, up-to-date responses.” Its help centre adds: “Each answer includes numbered citations linking to the original sources.” Those citations are the prize. A buyer who asks for a shortlist sees the sources beside the answer and can click through.
This sits on top of ordinary search work, which is why our GEO and AEO services cover technical access alongside content and evidence. If the terms are new to you, our comparison of AEO, GEO and SEO explains how they relate. Our guide to generative engine optimisation covers the research behind the term.
How Perplexity answers a question.
Perplexity reads the question, searches the web, and writes one answer from what it finds, with a link to each source it used. It does not answer from memory alone, so the pages it can reach matter. Our LLM SEO guide explains the two routes, training data and live retrieval.
Its help centre lists three parts: understanding your question, searching the web for “information from authoritative sources like articles, websites, and journals”, and summarising what it finds.
How hard it searches depends on the mode. Pro Search “conducts multiple searches across the web, drawing from articles, academic papers, forums, videos, and more”. The longer Research mode “performs dozens of searches automatically, reads hundreds of sources”.
So one buyer question becomes several narrower searches, and you compete to be the best available passage for each.
Buyer, in Perplexity: “Which payroll software suits a 120-person UK charity with term-time staff, and which vendors publish their implementation timelines?”
What decides whether a vendor is cited: the question splits into smaller ones (payroll for charities, term-time contracts, implementation time). A vendor is in the running only if PerplexityBot could fetch its pages, and only where a page has a short passage that answers one of those smaller questions on its own.
An invented question, written to show the pattern. It is not a real query or answer.
The two Perplexity crawlers.
Perplexity documents two user agents. PerplexityBot builds the search index and obeys robots.txt. Perplexity-User fetches a page when a person’s question needs it, and Perplexity says it generally does not follow robots.txt.
| User agent | What Perplexity says it does | robots.txt | Used for training? |
|---|---|---|---|
| PerplexityBot | “Designed to surface and link websites in search results on Perplexity.” | Followed. Perplexity recommends allowing it “to ensure your site appears in search results”. | “It is not used to crawl content for AI foundation models.” |
| Perplexity-User | “When users ask Perplexity a question, it might visit a web page to help provide an accurate answer and include a link to the page in its response.” | “Since a user requested the fetch, this fetcher generally ignores robots.txt rules.” | “Not used for web crawling or to collect content for training AI foundation models.” |
In server logs the user-agent strings contain PerplexityBot/1.0 and Perplexity-User/1.0. A user-agent string is easy to fake, so Perplexity also publishes the addresses each bot uses: a PerplexityBot address list and a Perplexity-User address list. When we read them on 12 November 2026 the first held 13 addresses and the second 10, and both files were dated 28 October 2026. The lists change, so always read the current files. Our guide to AI crawler log file analysis shows how to read those log lines.
Two more lines matter. “Each setting works independently”, and after a robots.txt change “it may take up to 24 hours for our systems to reflect changes.”
What robots.txt does and does not do.
Blocking PerplexityBot in robots.txt keeps the text of your pages out of Perplexity’s index, but it does not make your company disappear from Perplexity. Perplexity can still show your domain and headline, and describe you from other people’s pages.
Perplexity’s help article says: “Our crawler, PerplexityBot, will not index the full or partial text content of any site that disallows it via robots.txt. However, if a page is blocked, we may still index the domain, headline, and a brief factual summary.”
For a B2B company that is the worst of both outcomes: you can still be mentioned, but the detail has to come from a directory, a review site or a competitor’s comparison page.
Does allowing it mean training on our content?
Perplexity says no: “PerplexityBot indexes pages similarly to other search engines. Perplexity does not build foundation models, so your content will not be used for AI model pre-training.”
A robots.txt that keeps you in Perplexity
With no rule naming PerplexityBot and no blanket block, you are already allowed. To write the decision down, this is enough:
Allowing Perplexity’s search crawler (example)
# Perplexity search index
User-agent: PerplexityBot
Allow: /
Disallow: /admin/
# Everyone else
User-agent: *
Disallow: /admin/One detail catches people out, and it is why the private path appears twice in the example. The robots.txt standard (RFC 9309) says a crawler obeys the wildcard group only “if no matching group exists”. Once PerplexityBot has a group of its own, the rules under User-agent: * no longer apply to it, so any path it should avoid has to be repeated in its group.
The rules for other engines’ bots, and the question of whether an llms.txt file helps, are covered in our guide to llms.txt, schema and AI crawler rules.
The firewall layer.
A correct robots.txt is not enough if your firewall or content delivery network turns PerplexityBot away first. Sites block Perplexity this way without anyone having decided to.
Perplexity’s crawler documentation says: “If you’re using a Web Application Firewall (WAF) to protect your site, you may need to explicitly allow Perplexity’s bots to ensure they can access your content.” It gives steps for Cloudflare and AWS: match the user agent and the published addresses together, and place the allow rule ahead of any rule that blocks or challenges.
It includes one warning: “Cloudflare’s Bot Fight Mode can’t be skipped with WAF custom rules. If Bot Fight Mode is enabled, it can still challenge Perplexity bots that match this rule.”
Why this matters more for Perplexity than for most bots
On 1 July 2025 Cloudflare announced that it had become “the first Internet infrastructure provider to block AI crawlers accessing content without permission or compensation, by default.” A site on those defaults can be closed to AI crawlers without its owner choosing.
Then, on 4 August 2025, Cloudflare published a post saying it had observed Perplexity using undeclared crawlers to reach sites that had blocked it. Cloudflare said it had “de-listed them as a verified bot” and added blocking rules. Perplexity replied the same day and rejected the account, saying Cloudflare had confused its traffic with an unrelated cloud-browser service.
You do not need to settle that dispute. The practical point: if your site sits behind Cloudflare or a similar service, do not assume Perplexity’s declared crawlers are being let through. Check the bot settings, add the allow rule Perplexity describes, and confirm that PerplexityBot receives a normal page and not a challenge. Our short play on checking AI crawler access gives a procedure.
How Perplexity ranks what it has crawled.
Perplexity retrieves candidates by both exact wording and meaning, filters out stale or irrelevant ones, then re-ranks what is left in several stages, scoring passages as well as whole pages. It has not published a list of ranking factors.
What it has published is an engineering account. On 25 September 2025 Perplexity’s research team described the search system behind its products. It says the index “tracks over 200 billion unique URLs” and that the infrastructure “processes 200 million daily queries.”
- 01 · RETRIEVEKeyword and meaning searches run together and are merged into one candidate set
- 02 · PREFILTERClearly non-responsive or stale content is removed
- 03 · FAST RANKINGQuick keyword and embedding scores narrow the set
- 04 · RERANKHeavier reranking models decide the final order
- 05 · PASSAGESResults are scored at page level and at passage level
Source: Perplexity Research, 25 September 2025. The stage names are our shorthand.
Four details bear directly on how a page should be written.
- ✓Passages are ranked on their own. Perplexity says it breaks documents into “self-contained spans, each of which can be individually retrieved and ranked at query time.” A paragraph that needs the three before it is a weak candidate.
- ✓Exact words still count. Retrieval uses wording as well as meaning, so the product names, standards and job titles buyers type should appear on the page.
- ✓Staleness is filtered early. Stale content goes at the prefilter stage, and revisits are scheduled by a URL’s “importance and likely update frequency”.
- ✓Structure is kept. Its parser aims to capture content “in a manner that preserves the original content structure and layout”, and well-structured lists and tables suit simpler extraction rules.
When deciding which documents to keep in fast storage, Perplexity says it prioritises “documents from authoritative domains” and documents on “undercovered topics”. The second is encouraging for specialist B2B firms with a clear page on a narrow subject.
A caution: the article describes architecture, not a recipe with weights. For the writing side in detail, see our guide to the page patterns AI engines lift and credit.
Source labels.
Some Perplexity citations now carry a shield icon with one of three labels: Government, Academic or Trusted. Most company websites have none, and Perplexity says that is not a mark against them.
Perplexity’s help article on source labels describes a source review process that rates “the whole website a source comes from, not each individual page.” The review asks questions such as: “Does the site correct its mistakes? Does it say who wrote each piece? Does it keep news separate from advertising and opinion?”
| Label | What Perplexity says it marks | Its examples |
|---|---|---|
| Government | Official websites of government bodies. | nasa.gov, epa.gov |
| Academic | Scientific sites. | science.org, journals.plos.org |
| Trusted | A broad category for “sites that appear often enough in results to receive a label and that publish information within their own area of expertise.” | reuters.com, mayoclinic.org |
Two statements in that article matter for a marketing team:
- “Most domains on the web do not have a label. No label does not mean a site is low quality, and it is not a negative judgment.”
- “Perplexity’s partnerships, payments, and other business arrangements do not affect a site’s label.” A label cannot be bought.
The article does not say that labels change ranking, so we will not claim they do. What they show is the evidence Perplexity thinks a reader should weigh: named authors, visible corrections, and a clear line between editorial and advertising.
Going by Perplexity’s own examples, the sites that carry labels are government bodies, journals and established publishers, not vendors. Being described accurately on such sites is part of the work, and our guide to the off-site sources AI engines rely on for B2B firms ranks them by effort and value. For how often company websites themselves are cited, see our study of where AI answers get their sources.
Perplexity, ChatGPT and AI Overviews compared.
The three engines read the same web through three different front doors. Perplexity uses its own crawler and index, ChatGPT search uses OpenAI’s search crawler, and Google’s AI features use the ordinary Google index. Being open to one says nothing about the others.
| Perplexity | ChatGPT search | Google AI Overviews and AI Mode | |
|---|---|---|---|
| Crawler to allow | PerplexityBot | OAI-SearchBot | Googlebot |
| What gets you in | Being in Perplexity’s own index | OAI-SearchBot is “used to surface websites in search results in ChatGPT’s search features” | The page “must be indexed and eligible to be shown in Google Search with a snippet” |
| If you block the crawler | No page text indexed; domain, headline and a brief factual summary may remain | Not shown in ChatGPT search answers, “though can still appear as navigational links” | Controlled by the usual preview controls (nosnippet, max-snippet, noindex) |
| Fetch on a user’s request | Perplexity-User “generally ignores robots.txt rules” | For ChatGPT-User, “robots.txt rules may not apply” | Not covered in Google’s AI features page |
| Training crawler | None; Perplexity says it “does not build foundation models” | Separate: GPTBot | Separate control: Google-Extended |
Google’s column is the simplest. Its documentation says: “There are no additional requirements to appear in AI Overviews or AI Mode, nor other special optimizations necessary.” We set out how sources are chosen for AI Overviews in a separate guide. Our guide to Google AI Mode for B2B sites goes through that side.
ChatGPT is the closest to Perplexity in set-up, with one search crawler and one user-triggered fetcher. What it does with the pages it finds is covered in our explanation of how ChatGPT picks companies to recommend.
How much Perplexity crawls and refers.
PerplexityBot is a light crawler next to the training bots, and on the available data Perplexity has sent visitors back more often per page crawled than OpenAI or Anthropic. The figures are from 2025 and one network, so read them as direction only.
Cloudflare, a large content delivery network, published crawler data on 29 August 2025. In July 2025 PerplexityBot accounted for 0.4% of crawling by AI-only bots on its network. OpenAI’s training crawler, GPTBot, accounted for 28.1%.
The same post measured how many pages each company crawled for every visitor it referred back to a website. In July 2025 the figure was 38,065.7 for Anthropic, 1,091.4 for OpenAI, 194.8 for Perplexity and 5.4 for Google.
Perplexity’s ratio was moving the wrong way, though. Cloudflare wrote that “Perplexity decreased referrals in 2025”, with the ratio climbing from about 54 in January to about 195 in July.
For scale, Vercel, a website hosting company, reported on 17 December 2024 that PerplexityBot made 24.4 million fetches across its network in a month, against 569 million for GPTBot and 4.5 billion for Googlebot.
Two conclusions are safe. A crawler that visits this lightly has fewer chances to find your page, so a firewall challenge costs more. And Perplexity traffic will be a small line in your analytics, to be judged by what those visitors do. Our article on tying AI referral visits to pipeline shows how to measure that.
What a B2B site should change.
Fix access first, then make your key facts readable without JavaScript and quotable without context, then work on the sites that describe you. There is no point polishing a passage Perplexity cannot fetch.
1. Open the door on purpose
Allow PerplexityBot in robots.txt, add the firewall rule from Perplexity’s documentation, and check whether bot protection is challenging it anyway. Then allow up to 24 hours for the change to register.
2. Put the words in the HTML
Vercel’s December 2024 study found that “none of the major AI crawlers currently render JavaScript”, and named PerplexityBot among them. The data is nearly two years old and Perplexity has not documented the point, so treat it as a risk to remove. View the source of a service page and search for a sentence from the middle of it. If it is missing, see our play on server-rendered content.
3. Write passages that stand alone
Each key fact needs a passage that names its subject and makes sense when lifted out. “It takes six weeks” fails. “Implementation for a 100 to 250 seat organisation usually takes six weeks” can be quoted. Put the question in the heading and the answer in the first sentence beneath it. Use your buyers’ exact terms, and a real table or list for specifications and steps.
4. Keep pages current and honestly dated
Stale content is filtered before ranking. Review commercial pages on a schedule and change the visible date only when the content changes.
5. Check what Perplexity already says
Ask Perplexity the questions your buyers ask, open the sources panel and write down which sites are cited. Our method for checking how AI engines answer buyer questions gives you a repeatable way to do it. Or have the first pass done for you: the free AI visibility report shows who is named and which sources are cited.
The gaps usually fall into three groups: pages that could not be read, pages with nothing quotable, and third-party sites that describe competitors and not you. Work through them in that order. If you want help getting recommended in AI answers, our scope covers technical access, content and evidence.
The crawler-access self-check.
Paste your robots.txt file to see how Perplexity’s crawlers are treated, then tick the statements that are true for your site. Everything runs in your browser; nothing is sent anywhere.
Perplexity readiness: 0 / 100
Tick what is true today. If you have not checked, leave it unticked.
The paste check cannot see your firewall, so the second and third ticks matter as much as the first.
Questions owners ask.
What is Perplexity SEO?
It is the work of making your pages easy for Perplexity to crawl, retrieve and cite: allowing PerplexityBot through robots.txt and your firewall, keeping key text in the HTML, and writing passages that make sense on their own.
How do I rank in Perplexity?
Perplexity has not published ranking factors. Its research team describes a pipeline that retrieves by wording and meaning, filters stale content, and re-ranks passages as well as pages. So be crawlable, be current, and give each important fact a self-contained passage.
Does Perplexity respect robots.txt?
Perplexity says PerplexityBot does: it “will not index the full or partial text content of any site that disallows it via robots.txt.” Its user-triggered fetcher, Perplexity-User, “generally ignores robots.txt rules” because a person requested the page. Cloudflare disputed this in August 2025; Perplexity rejected its account.
If we block PerplexityBot, do we vanish from Perplexity?
No. Perplexity says that when a page is blocked it “may still index the domain, headline, and a brief factual summary.” You lose control of the detail, not the mention.
Does Perplexity use Google’s index?
Not according to Perplexity. Its research team wrote in September 2025 that the first version used third-party search services and that the company then built its own index. A page can rank well in Google and still be missing from Perplexity if PerplexityBot cannot reach it.
Sources: Perplexity crawlers; Perplexity help centre: robots.txt; how Perplexity works; answer engines; Pro Search; source labels; Perplexity Research, 25 Sep 2025; Perplexity blog, 4 Aug 2025; Cloudflare blog, 4 Aug 2025; RFC 9309; Cloudflare press release, 1 Jul 2025; Cloudflare blog, 29 Aug 2025; Vercel, 17 Dec 2024; OpenAI, crawlers; Google, AI features. All accessed 12 November 2026.