Category
Tech stack
AI readiness
AI training policy
reserved
AI-bot protection
Blocked AI bots
cohere-aiYouBotGoogle-Extendedanthropic-aiCCBotBytespiderDiffbotOmgilibotPerplexityBotGPTBotChatGPT-UserOAI-SearchBotClaude-WebMeta-ExternalAgentAmazonbotTimpibot · 16
AI files
Evidence
robots:cohere-ai+YouBot+Google-Extended+anthropic-ai+CCBot+Bytespider+Diffbot+Omgilibot+PerplexityBot+GPTBot+ChatGPT-User+OAI-SearchBot+Claude-Web+Meta-ExternalAgent+Amazonbot+Timpibotllms.txt
Compliance (GEO / GDPR)
TLD