Bot traffic is nothing new – crawlers have been part of the web for 30 years. What’s new is how difficult it has become to detect. And how large their share is: in June 2026, Cloudflare measured 57.5 percent machine-based traffic, Imperva recorded 53 percent for 2025.
The trend has developed over the years but has accelerated dramatically following the commercial breakthrough of large language models.
Bots are getting harder to spot
Classic crawlers are easy to identify: a specific and valid user agent, usually honest self-disclosure, and no JavaScript. What’s now sweeping across websites looks very different. Messenger Services fetch every shared article for link previews. Headless browsers execute scripts, load images, and set cookies. AI agents retrieve texts on behalf of a user. And language models collect data for training – without a user behind them, systematically across the entire content, including archives. Many of these actions are now barely distinguishable from human traffic.
Noise turns into a roar
What is crucial here is not just the volume, but the highly uneven distribution across the content of a given site. The machine-generated noise does not spread evenly across the data – and therefore distorts any analysis. Article rankings shift, retention time is skewed, and an A/B test might end up measuring machines more than people.
Where humans end and machines begin
This raises a question for Analytics that simply didn’t exist with such urgency a few years ago: What counts as a reader?
Our answer: Any agent reading on behalf of a user is a machine – so, a bot. The same goes for any access that clearly identifies itself as a crawler or bot.
Clear data for clear decisions
We’ve decided not to simply filter out this grey area, but to make it visible. Upscore now separates machine from human traffic – selectable in the dashboard. Nothing is blocked; everything is reported.
Behind this is our AI-based bot detection, which evaluates requests based on a wide range of signals, also identifying automated traffic that doesn’t behave like a classic bot. This ensures that it remains clear what real readers are reading, clicking, and using, allowing editorial, product and sales teams to make decisions based on actual human traffic.
Peer Weise is Business Solutions Manager at Upscore