Anthropic · AI crawler
2,102,742 websites block ClaudeBot
That’s 4.332% of the 48,534,257 live websites we index — the #4 most-blocked AI crawler.
Model trainingRespects robots.txtAI Data Scrapers
Scrapes data to train LLMs and AI products offered by Anthropic.
Official documentation ↗
🛰️ Observed crawling piperic.com unverified
100,117 requests · last seen 10m ago · 2 IP(s)
Fetched: HTML (68364) · other (31746) · robots.txt (4) · sitemap (3)
Block ClaudeBot in robots.txt
Add this to your site’s /robots.txt:
User-agent: ClaudeBot
Disallow: /
Who blocks ClaudeBot
Top countries blocking it
Top categories blocking it
← All AI bots