Category
AI readiness
AI training policy
reserved
AI-bot protection
Blocked AI bots
Meta-ExternalAgentPerplexityBotApplebot-ExtendedBytespiderClaude-WebGPTBotAmazonbotCCBotClaudeBotGoogle-ExtendedOAI-SearchBotOmgilibotanthropic-aicohere-aiDiffbotImagesiftBotChatGPT-UserFacebookBotTimpibotYouBot · 20
AI files
Evidence
robots:Meta-ExternalAgent+PerplexityBot+Applebot-Extended+Bytespider+Claude-Web+GPTBot+Amazonbot+CCBot+ClaudeBot+Google-Extended+OAI-SearchBot+Omgilibot+anthropic-ai+cohere-ai+Diffbot+ImagesiftBot+ChatGPT-User+FacebookBot+Timpibot+YouBot
Compliance (GEO / GDPR)
TLD