By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
Online Tech Guru
  • News
  • PC/Windows
  • Mobile
  • Apps
  • Gadgets
  • More
    • Gaming
    • Accessories
    • Editor’s Choice
    • Press Release
Reading: Cloudflare says Perplexity’s AI bots are ‘stealth crawling’ blocked sites
Best Deal
Font ResizerAa
Online Tech GuruOnline Tech Guru
  • News
  • Mobile
  • PC/Windows
  • Gaming
  • Apps
  • Gadgets
  • Accessories
Search
  • News
  • PC/Windows
  • Mobile
  • Apps
  • Gadgets
  • More
    • Gaming
    • Accessories
    • Editor’s Choice
    • Press Release

Top 10 Best-Selling Magic The Gathering Edge of Eternities Cards of 2025 (So Far)

News Room News Room 4 August 2025
FacebookLike
InstagramFollow
YoutubeSubscribe
TiktokFollow
  • Subscribe
  • Privacy Policy
  • Contact
  • Terms of Use
© Foxiz News Network. Ruby Design Company. All Rights Reserved.
Online Tech Guru > News > Cloudflare says Perplexity’s AI bots are ‘stealth crawling’ blocked sites
News

Cloudflare says Perplexity’s AI bots are ‘stealth crawling’ blocked sites

News Room
Last updated: 4 August 2025 20:41
By News Room 3 Min Read
Share
SHARE

The AI search startup Perplexity is allegedly skirting restrictions meant to stop its AI web crawlers from accessing certain websites, according to a report from Cloudflare. In the report, Cloudflare claims that when Perplexity encounters a block, the startup will conceal its crawling identity “in an attempt to circumvent the website’s preferences.”

The report only adds to concerns about Perplexity vacuuming up content without permission, as the company got caught barging past paywalls and ignoring sites’ robots.txt files last year. At the time, Perplexity CEO Aravind Srinivas blamed the activity on third-party crawlers used by the site.

Now, Cloudflare, one of the world’s biggest internet architecture providers, says it received complaints from customers who claimed that Perplexity’s bots still had access to their websites even after putting their preference in their websites’ robots.txt file and by creating Web Application Firewall (WAF) rules to restrict access to the startup’s AI bots.

To test this, Cloudflare says it created new domains with similar restrictions against Perplexity’s AI scrapers. It found that the startup will first attempt to access the sites by identifying itself as the names of its crawlers: “PerplexityBot” or “Perplexity-User.”

But if the website has restrictions against AI scraping, Cloudflare claims Perplexity will change its user agent — the bit of information that tells a website what kind of browser and device you’re using, or if the visitor is a bot — to “impersonate Google Chrome on macOS.” Cloudflare says this “undeclared crawler” uses “rotating” IP addresses that the company doesn’t include on the list of IP addresses used by its bots.

Additionally, Cloudflare claims that Perplexity changes its autonomous system networks (ASN), a number used to identify groups of IP networks controlled by a single operator, to get around blocks as well. “This activity was observed across tens of thousands of domains and millions of requests per day,” Cloudflare writes.

In a statement to The Verge, Perplexity spokesperson Jesse Dwyer called Cloudflare’s report a “publicity stunt,” adding that “there are a lot of misunderstandings in the blog post.” Cloudflare has since de-listed Perplexity as a verified bot and has rolled out methods to block Perplexity’s “stealth crawling.”

Share This Article
Facebook Twitter Copy Link
Leave a comment

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Valve refutes Mastercard’s denial it has not pressured game platforms over NSFW content

News Room News Room 4 August 2025
FacebookLike
InstagramFollow
YoutubeSubscribe
TiktokFollow

Trending

Best cheap Kindle deals of August 2025

When it comes to finding a device to read ebooks, you have a few options…

4 August 2025

Silent Hill f Is Combat Heavy Because ‘Challenging Action Games Are Gaining Popularity Among Younger Players Nowadays’

We've already heard a lot about how Silent Hill f hopes to set itself apart…

4 August 2025

Poco Teases Arrival of New Smartphone; Poco M7 Plus India Launch Date, Price Range Tipped

Poco is preparing to unveil a new smartphone in India, as teased on its official…

4 August 2025
News

Amazon’s best Kindles are cheaper than ever at Best Buy

A Kindle lets you carry an entire bookshelf while on the go, which is why almost every year we recommend the e-reader in our guides to the best back to…

News Room 4 August 2025

Your may also like!

News

One of the biggest newsletter platforms now syndicates to Bluesky and Mastodon

News Room 4 August 2025
Mobile

Samsung Galaxy S25 FE Design, Colourways Leaked Online

News Room 4 August 2025
News

Science Reveals the Surprising Origins of the Potato

News Room 4 August 2025
News

Alienware’s AW2725Q 4K OLED gaming monitor is down to its lowest price ever at Amazon

News Room 4 August 2025

Our website stores cookies on your computer. They allow us to remember you and help personalize your experience with our site.

Read our privacy policy for more information.

Quick Links

  • Subscribe
  • Privacy Policy
  • Contact
  • Terms of Use
Advertise with us

Socials

Follow US
Welcome Back!

Sign in to your account

Lost your password?