In short. Cloudflare associates Bytespider with ByteDance, documents its identification strings and reports historical changes in its observed crawling traffic.

Definition: Bytespider is a crawler name that Cloudflare associates with ByteDance. Cloudflare Radar lists user agents containing Bytespider under its verified Toutiao search bot, while Cloudflare's AI Crawl Control reference classifies Bytespider as an AI crawler.

Key facts

As of 17 October 2026, Cloudflare Radar's Toutiao directory entry describes an automated news aggregation bot and displays HTTP user agents containing Bytespider.
Cloudflare's Bot reference, updated 23 April 2026, lists ByteDance as the operator, Bytespider as the user agent and 33563853 as the detection ID.
In its July 2025 crawler analysis, Cloudflare reports an 85% decline in Bytespider request volume between May 2024 and May 2025.
The IETF's Robots Exclusion Protocol defines robots.txt rules as instructions that crawlers are requested to honor, rather than access authorization.

How is Bytespider identified?

A user agent is an identification string sent with a web request. Radar's verified Toutiao record provides this example:

Mozilla/5.0 (compatible; Bytespider; https://zhanzhang.toutiao.com/) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.0.0 Safari/537.36

The record names Bytedance as operator, assigns the Search category and describes direct access, meaning that the operator controls the bot's activity. It also links to Toutiao's webmaster documentation.

Cloudflare's separate AI Crawl Control reference places Bytespider in the AI Crawler category. These source labels matter when describing a request: the Bytespider string appears in documentation covering both search and AI crawling.

Cloudflare's IP validation documentation, updated 15 April 2026, describes verification using an operator's public IP list or domain-based reverse DNS, which checks the domain name associated with an IP address. A user-agent match alone therefore provides less identity evidence than this verification process.

What traffic does Cloudflare observe?

Cloudflare's analysis published on 1 July 2025 compares a cohort of more than 30 AI and search crawlers observed on its network. Within that cohort, Bytespider's share falls from 22.8% in May 2024 to 2.9% in May 2025. Its raw request volume falls by 85% over the same comparison.

The article also presents an AI-only comparison based on crawler names matched against user-agent strings in HTTP requests. In that narrower group, its table gives Bytespider a 42% share in May 2024 and 7.2% in May 2025. The denominator must accompany either set of figures: AI-only traffic and the combined crawler cohort produce different shares.

These figures describe historical traffic observed on Cloudflare's network within the stated crawler groups.

What controls are documented?

Cloudflare's Bot reference says Bot Management customers can use cf.bot_management.detection_ids in web application firewall rules. It also documents user-agent matching in robots.txt or firewall rules for other plans.

The robots.txt file communicates crawling rules for URL paths. Under RFC 9309, access protection requires a security measure appropriate to the application. For a website deciding how to handle Bytespider requests, the distinction separates a crawling instruction from a firewall's access control.

FAQ

Which company operates Bytespider?

Cloudflare's AI Crawl Control reference identifies ByteDance as the operator. Radar uses the spelling Bytedance in its Toutiao record.

Which traffic percentage should I use?

Use the percentage with its measurement period and crawler group. Cloudflare's July 2025 analysis reports different shares for AI-only traffic and its combined AI and search crawler cohort.

Sources

Check how a page reads to answer engines. Open the GEO score analyzer