Category
Tech stack
CMS
Analytics
CDN
Framework
AI readiness
AI training policy
reserved
AI-bot protection
Blocked AI bots
DiffbotOmgilibotanthropic-aiMeta-ExternalAgentPerplexityBotApplebot-ExtendedBytespiderChatGPT-UserClaude-WebImagesiftBotAmazonbotGPTBotGoogle-ExtendedFacebookBotYouBotCCBotClaudeBot · 17
AI files
Evidence
robots:Diffbot+Omgilibot+anthropic-ai+Meta-ExternalAgent+PerplexityBot+Applebot-Extended+Bytespider+ChatGPT-User+Claude-Web+ImagesiftBot+Amazonbot+GPTBot+Google-Extended+FacebookBot+YouBot+CCBot+ClaudeBot
Compliance (GEO / GDPR)
TLD