Category
AI readiness
AI training policy
reserved
AI-bot protection
Blocked AI bots
Applebot-ExtendedClaude-WebYouBotGoogle-ExtendedDiffbotanthropic-aiClaudeBotCCBotTimpibotOmgilibotAmazonbotMeta-ExternalAgentPerplexityBotcohere-aiBytespiderFacebookBot · 16
AI files
Evidence
robots:Applebot-Extended+Claude-Web+YouBot+Google-Extended+Diffbot+anthropic-ai+ClaudeBot+CCBot+Timpibot+Omgilibot+Amazonbot+Meta-ExternalAgent+PerplexityBot+cohere-ai+Bytespider+FacebookBot
Compliance (GEO / GDPR)
TLD
Overview
Title
Ars Technica - Serving the Technologist since 1998. News, reviews, and analysis.
Description
News and reviews, covering IT, AI, science, space, health, gaming, cybersecurity, tech policy, computers, mobile devices, and operating systems.
Final URL
Language
en (html)
Scanned at
2026-06-18 22:47:41