Category
AI readiness
AI training policy
reserved
AI-bot protection
Blocked AI bots
anthropic-aiFacebookBotAmazonbotApplebot-ExtendedChatGPT-UserOAI-SearchBotYouBotBytespidercohere-aiDiffbotOmgilibotTimpibotPerplexityBotCCBotClaudeBotClaude-WebGoogle-ExtendedGPTBotImagesiftBotMeta-ExternalAgent · 20
AI files
Evidence
robots:anthropic-ai+FacebookBot+Amazonbot+Applebot-Extended+ChatGPT-User+OAI-SearchBot+YouBot+Bytespider+cohere-ai+Diffbot+Omgilibot+Timpibot+PerplexityBot+CCBot+ClaudeBot+Claude-Web+Google-Extended+GPTBot+ImagesiftBot+Meta-ExternalAgent
Compliance (GEO / GDPR)
TLD
Overview
Title
The New York Times - Breaking News, US News, World News and Videos
Description
Live news, investigations, opinion, photos and video by the journalists of The New York Times from more than 150 countries around the world. Subscribe for coverage of U.S. and international news, politics, business, technology, science, health, arts, sports and more.
Final URL
Language
en (html)
Scanned at
2026-06-25 11:14:05