Category
Tech stack
AI readiness
AI training policy
reserved
AI-bot protection
Blocked AI bots
CCBotDiffbotMeta-ExternalAgentOAI-SearchBotPerplexityBotApplebot-ExtendedBytespiderGPTBotImagesiftBotClaude-WebOmgilibotAmazonbotClaudeBotcohere-aiFacebookBotGoogle-Extendedanthropic-aiChatGPT-User · 18
AI files
Evidence
robots:CCBot+Diffbot+Meta-ExternalAgent+OAI-SearchBot+PerplexityBot+Applebot-Extended+Bytespider+GPTBot+ImagesiftBot+Claude-Web+Omgilibot+Amazonbot+ClaudeBot+cohere-ai+FacebookBot+Google-Extended+anthropic-ai+ChatGPT-User
Compliance (GEO / GDPR)
TLD