Category
Tech stack
Framework
AI readiness
AI training policy
reserved
AI-bot protection
Blocked AI bots
Meta-ExternalAgentOAI-SearchBotanthropic-aiCCBotDiffbotBytespiderClaude-WebGPTBotcohere-aiGoogle-ExtendedYouBotAmazonbotApplebot-ExtendedClaudeBotTimpibotChatGPT-UserFacebookBotPerplexityBot · 18
AI files
Evidence
robots:Meta-ExternalAgent+OAI-SearchBot+anthropic-ai+CCBot+Diffbot+Bytespider+Claude-Web+GPTBot+cohere-ai+Google-Extended+YouBot+Amazonbot+Applebot-Extended+ClaudeBot+Timpibot+ChatGPT-User+FacebookBot+PerplexityBot
Compliance (GEO / GDPR)
TLD