By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
Online Tech Guru
  • News
  • PC/Windows
  • Mobile
  • Apps
  • Gadgets
  • More
    • Gaming
    • Accessories
    • Editor’s Choice
    • Press Release
Reading: Cloudflare says Perplexity’s AI bots are ‘stealth crawling’ blocked sites
Best Deal
Font ResizerAa
Online Tech GuruOnline Tech Guru
  • News
  • Mobile
  • PC/Windows
  • Gaming
  • Apps
  • Gadgets
  • Accessories
Search
  • News
  • PC/Windows
  • Mobile
  • Apps
  • Gadgets
  • More
    • Gaming
    • Accessories
    • Editor’s Choice
    • Press Release

The Best OLED TVs

News Room News Room 29 August 2025
FacebookLike
InstagramFollow
YoutubeSubscribe
TiktokFollow
  • Subscribe
  • Privacy Policy
  • Contact
  • Terms of Use
© Foxiz News Network. Ruby Design Company. All Rights Reserved.
Online Tech Guru > News > Cloudflare says Perplexity’s AI bots are ‘stealth crawling’ blocked sites
News

Cloudflare says Perplexity’s AI bots are ‘stealth crawling’ blocked sites

News Room
Last updated: 4 August 2025 20:41
By News Room 3 Min Read
Share
SHARE

The AI search startup Perplexity is allegedly skirting restrictions meant to stop its AI web crawlers from accessing certain websites, according to a report from Cloudflare. In the report, Cloudflare claims that when Perplexity encounters a block, the startup will conceal its crawling identity “in an attempt to circumvent the website’s preferences.”

The report only adds to concerns about Perplexity vacuuming up content without permission, as the company got caught barging past paywalls and ignoring sites’ robots.txt files last year. At the time, Perplexity CEO Aravind Srinivas blamed the activity on third-party crawlers used by the site.

Now, Cloudflare, one of the world’s biggest internet architecture providers, says it received complaints from customers who claimed that Perplexity’s bots still had access to their websites even after putting their preference in their websites’ robots.txt file and by creating Web Application Firewall (WAF) rules to restrict access to the startup’s AI bots.

To test this, Cloudflare says it created new domains with similar restrictions against Perplexity’s AI scrapers. It found that the startup will first attempt to access the sites by identifying itself as the names of its crawlers: “PerplexityBot” or “Perplexity-User.”

But if the website has restrictions against AI scraping, Cloudflare claims Perplexity will change its user agent — the bit of information that tells a website what kind of browser and device you’re using, or if the visitor is a bot — to “impersonate Google Chrome on macOS.” Cloudflare says this “undeclared crawler” uses “rotating” IP addresses that the company doesn’t include on the list of IP addresses used by its bots.

Additionally, Cloudflare claims that Perplexity changes its autonomous system networks (ASN), a number used to identify groups of IP networks controlled by a single operator, to get around blocks as well. “This activity was observed across tens of thousands of domains and millions of requests per day,” Cloudflare writes.

In a statement to The Verge, Perplexity spokesperson Jesse Dwyer called Cloudflare’s report a “publicity stunt,” adding that “there are a lot of misunderstandings in the blog post.” Cloudflare has since de-listed Perplexity as a verified bot and has rolled out methods to block Perplexity’s “stealth crawling.”

Share This Article
Facebook Twitter Copy Link
Leave a comment

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

NBA 2K26 Review in Progress

News Room News Room 29 August 2025
FacebookLike
InstagramFollow
YoutubeSubscribe
TiktokFollow

Trending

How to See the Total Lunar Eclipse and Blood Moon on September 7

On the evening of September 7, the second (and final) total lunar eclipse of the…

29 August 2025

TikTok is now letting everyone DM each other with voice memos and pictures

Every platform wants to be the place you hang with friends — even Spotify as…

29 August 2025

Why are Switch 2 dev kits so hard to get? | Opinion

Nintendo’s Switch 2 has leapt out of the gates at a stunning pace. Circana’s latest…

29 August 2025
News

Microsoft’s next annual update for Windows 11 is in Release Preview testing

So, will you see new UI features or more AI tweaks included on your desktop with this update? Microsoft didn’t mention many specific features or changes in the update, other…

News Room 29 August 2025

Your may also like!

News

Review: DJI Mic 3

News Room 29 August 2025
News

The Best Labor Day Deals and Sales

News Room 29 August 2025
News

Hisense’s take on the Samsung Frame TV is $300 off

News Room 29 August 2025
Gaming

Metal Gear Solid Delta: Snake Eater Already Has Essential Mods That Improve PC Performance, Fix Graphics, and Let You Play as Hideo Kojima

News Room 29 August 2025

Our website stores cookies on your computer. They allow us to remember you and help personalize your experience with our site.

Read our privacy policy for more information.

Quick Links

  • Subscribe
  • Privacy Policy
  • Contact
  • Terms of Use
Advertise with us

Socials

Follow US
Welcome Back!

Sign in to your account

Lost your password?