Category
Tech stack
AI readiness
AI training policy
reserved
AI-bot protection
Blocked AI bots
Google-ExtendedOmgilibotAmazonbotApplebot-ExtendedBytespiderCCBotFacebookBotanthropic-aiClaudeBotGPTBotMeta-ExternalAgentPerplexityBot · 12
AI files
Evidence
robots:Google-Extended+Omgilibot+Amazonbot+Applebot-Extended+Bytespider+CCBot+FacebookBot+anthropic-ai+ClaudeBot+GPTBot+Meta-ExternalAgent+PerplexityBot
Compliance (GEO / GDPR)
TLD
Overview
Title
Scott McLemee – digital repository & public notebook
Description
digital repository & public notebook
Final URL
Language
en (html)
Scanned at
2026-06-24 16:01:02