Cloudflare, Inc. (NYSE: NET) says AI agents and other automated visitors now account for more than half of the traffic reaching sites on its network, forcing businesses to reconsider a long-standing assumption of the web: a visitor is a person who can create advertising, subscription or sales value. The company’s network research points to a more immediate operational question for IT and digital-business teams: which automated requests are customers, which are data collection, and which simply add cost.
Cloudflare said it processed nearly 115 million HTTP requests per second at the time of its analysis, up from an average of 63 million requests per second at the end of 2024. Daily requests from AI agents on its network grew by more than 1,700% over the past year, according to the company. Those figures reflect Cloudflare’s network rather than the entire internet, but they show the scale of the planning problem for organizations that operate content, commerce, documentation or data services online.
For years, many websites accepted search-engine crawling because a crawler could lead to a human visit. Cloudflare argues that AI answer engines can change that exchange: a system can collect page content and return a summary without sending a reader to the publisher’s site. The request still consumes bandwidth, compute and origin capacity, but it may not produce a referral, ad impression or subscription.
The company said some heavily crawled categories, including retail, computer software, IT services and financial services, have seen human traffic fall by as much as 40% in less than a year. Cloudflare did not present that observation as a universal result for all businesses. It nevertheless gives digital leaders a reason to measure automated traffic separately from human sessions before treating rising request volume as evidence of stronger customer demand.
Not every nonhuman request has the same value. A training crawler collects content for a model, while an agent may return repeatedly because a person asked a question, compared products or initiated a transaction. Cloudflare’s research distinguishes those uses from traditional search crawling, suggesting that businesses need policies based on a request’s purpose and potential commercial value rather than a single rule for all bots.
That distinction is difficult when an operator cannot reliably identify what is making a request. Cloudflare said it sees more than 500 billion verified bot requests each week through Web Bot Auth, a mechanism that uses cryptographic signatures to help a site distinguish an authentic agent from an impersonator. For enterprise teams, the broader lesson is that bot governance increasingly depends on identity, observability and traffic classification, not just IP-address lists and user-agent strings.
Cloudflare also reported that fewer than 1% of sites on its network block search crawlers, while 17% block training crawlers. The split illustrates why a blanket ban on automated traffic may be too blunt: businesses may still want search discovery or agent-driven transactions while declining to provide content for model training. A policy that cannot separate those cases risks blocking prospective customers along with unwanted collection.
Cloudflare is positioning separate search, agent and training controls, plus pay-per-use and monetization products, as tools for handling the transition. Those offerings are still developing, and a business will need to assess their pricing, compatibility and commercial terms before relying on them. The research matters even for organizations that do not use Cloudflare because it frames a more general issue: automated access is becoming a distinct traffic class with its own cost, security and revenue implications.
For CIOs and digital leaders, the practical response is to make agent traffic visible in web analytics, identify which automated visitors can access which content or actions, and test whether high-cost requests lead to meaningful business outcomes. Legacy web metrics treated page views and crawler activity as supporting indicators of human reach. As AI agents become a second audience, businesses will need to measure and govern the machine visitor as deliberately as the human one.