Category
AI readiness
AI training policy
reserved
AI-bot protection
Blocked AI bots
CCBotFacebookBotYouBotBytespiderGoogle-ExtendedOAI-SearchBotOmgilibotPerplexityBotcohere-aiDiffbotTimpibotClaudeBotGPTBotImagesiftBotAmazonbotanthropic-aiApplebot-ExtendedChatGPT-UserClaude-WebMeta-ExternalAgent · 20
AI files
Evidence
robots:CCBot+FacebookBot+YouBot+Bytespider+Google-Extended+OAI-SearchBot+Omgilibot+PerplexityBot+cohere-ai+Diffbot+Timpibot+ClaudeBot+GPTBot+ImagesiftBot+Amazonbot+anthropic-ai+Applebot-Extended+ChatGPT-User+Claude-Web+Meta-ExternalAgent