Category
Tech stack
Framework
AI readiness
AI training policy
reserved
AI-bot protection
Blocked AI bots
ClaudeBotDiffbotFacebookBotAmazonbotBytespiderClaude-Webcohere-aiGoogle-ExtendedTimpibotApplebot-ExtendedCCBotGPTBotPerplexityBotYouBotanthropic-aiChatGPT-UserMeta-ExternalAgentOAI-SearchBot · 18
AI files
Evidence
robots:ClaudeBot+Diffbot+FacebookBot+Amazonbot+Bytespider+Claude-Web+cohere-ai+Google-Extended+Timpibot+Applebot-Extended+CCBot+GPTBot+PerplexityBot+YouBot+anthropic-ai+ChatGPT-User+Meta-ExternalAgent+OAI-SearchBot
Compliance (GEO / GDPR)
TLD