Category
Primary
spam-or-harmful-content · embed-ft 0.44
Tree path
Sensitive Topics › Spam or Harmful Content
Group (tier-1)
Tech stack
CDN
AI readiness
AI training policy
reserved
AI-bot protection
Blocked AI bots
PerplexityBotClaudeBotDiffbotGoogle-ExtendedImagesiftBotOAI-SearchBotcohere-aiFacebookBotGPTBotYouBotAmazonbotanthropic-aiBytespiderCCBotClaude-WebMeta-ExternalAgentTimpibot · 17
AI files
Evidence
robots:PerplexityBot+ClaudeBot+Diffbot+Google-Extended+ImagesiftBot+OAI-SearchBot+cohere-ai+FacebookBot+GPTBot+YouBot+Amazonbot+anthropic-ai+Bytespider+CCBot+Claude-Web+Meta-ExternalAgent+Timpibot
Compliance (GEO / GDPR)
TLD