Category
Tech stack
CMS
Analytics
CDN
Framework
CRM/marketing
AI readiness
AI training policy
reserved
Terms of Service
AI-bot protection
Blocked AI bots
GPTBotanthropic-aiBytespiderOmgilibotCCBotAmazonbotYouBotApplebot-ExtendedPerplexityBotMeta-ExternalAgentcohere-aiOAI-SearchBotChatGPT-UserClaudeBotGoogle-ExtendedDiffbotTimpibotFacebookBotImagesiftBot · 19
AI files
Evidence
robots:GPTBot+anthropic-ai+Bytespider+Omgilibot+CCBot+Amazonbot+YouBot+Applebot-Extended+PerplexityBot+Meta-ExternalAgent+cohere-ai+OAI-SearchBot+ChatGPT-User+ClaudeBot+Google-Extended+Diffbot+Timpibot+FacebookBot+ImagesiftBotllms.txt
Compliance (GEO / GDPR)
TLD