Category
Tech stack
CDN
AI readiness
AI training policy
prohibited
AI-bot protection
Blocked AI bots
ClaudeBotDiffbotMeta-ExternalAgentYouBotApplebot-ExtendedBytespiderClaude-WebGPTBotGoogle-ExtendedPerplexityBotanthropic-aiAmazonbotChatGPT-UserOmgilibotFacebookBotOAI-SearchBotTimpibotcohere-aiCCBotImagesiftBot · 20
AI files
Evidence
robots:ClaudeBot+Diffbot+Meta-ExternalAgent+YouBot+Applebot-Extended+Bytespider+Claude-Web+GPTBot+Google-Extended+PerplexityBot+anthropic-ai+Amazonbot+ChatGPT-User+Omgilibot+FacebookBot+OAI-SearchBot+Timpibot+cohere-ai+CCBot+ImagesiftBotmeta:noai
Compliance (GEO / GDPR)
TLD