Category
Primary
web-development · embed-ft 0.46
Tree path
Technology & Computing › Computing › Internet › Web Development
Group (tier-1)
Also (top-3)
AI readiness
AI training policy
reserved
AI-bot protection
Blocked AI bots
anthropic-aiClaudeBotGPTBotOmgilibotTimpibotBytespiderChatGPT-UserImagesiftBotGoogle-ExtendedOAI-SearchBotPerplexityBotYouBotApplebot-ExtendedClaude-Webcohere-aiFacebookBotMeta-ExternalAgentAmazonbotCCBotDiffbot · 20
AI files
Evidence
robots:anthropic-ai+ClaudeBot+GPTBot+Omgilibot+Timpibot+Bytespider+ChatGPT-User+ImagesiftBot+Google-Extended+OAI-SearchBot+PerplexityBot+YouBot+Applebot-Extended+Claude-Web+cohere-ai+FacebookBot+Meta-ExternalAgent+Amazonbot+CCBot+Diffbot
Compliance (GEO / GDPR)
TLD