ByteDance · AI crawler
2,099,388 websites block Bytespider
That’s 4.326% of the 48,534,257 live websites we index — the #5 most-blocked AI crawler.
Model trainingIgnores robots.txtAI Data Scrapers
Downloads data to train LLMS, including ChatGPT competitors.
🛰️ Observed crawling piperic.com ✓ verified
33 requests · last seen 31m ago · 30 IP(s)
Fetched: other (18) · HTML (14) · robots.txt (1)
Block Bytespider in robots.txt
Add this to your site’s /robots.txt:
User-agent: Bytespider
Disallow: /
Who blocks Bytespider
Top countries blocking it
Top categories blocking it
← All AI bots