OpenAI · AI crawler
2,232,106 websites block GPTBot
That’s 4.599% of the 48,534,257 live websites we index — the #1 most-blocked AI crawler.
Model trainingRespects robots.txtAI Data Scrapers
Data is used to train current and future models, removed paywalled data, PII and data that violates the company's policies.
Official documentation ↗
🛰️ Observed crawling piperic.com unverified✅ uses llms.txt
1,901 requests · last seen 12m ago · 2 IP(s)
Fetched: HTML (1692) · other (207) · llms.txt (2)
Block GPTBot in robots.txt
Add this to your site’s /robots.txt:
User-agent: GPTBot
Disallow: /
Who blocks GPTBot
Top countries blocking it
Top categories blocking it
← All AI bots