Check, live, which AI crawlers your site blocks or allows — GPTBot, ClaudeBot, PerplexityBot, Google-Extended and more — whether you publish llms.txt, and if you’re visible to ChatGPT, Claude & Perplexity.
Free · no signup · live check against the site’s robots.txt and AI-policy signals.
Ever wondered who — or what — is reading your website behind the scenes? While human traffic is great, a massive wave of automated AI bots is constantly sweeping the web, aggressively hunting for fresh data to train the next generation of Large Language Models (LLMs). Whether you want to feed these models to boost your brand’s AI visibility, or keep your intellectual property strictly locked down, you need to know exactly who you’re letting in.
Our live AI Crawler Checker instantly reveals your site’s relationship with the biggest AI platforms. With a single click, you can audit your robots.txt directives and AI-policy signals to see whether you are actively blocking or allowing access to major data scrapers.
A real-time check against the most active AI user-agents currently crawling the web:
We don’t just look for basic disallow rules. Our checker also detects the presence of llms.txt — a rapidly growing web standard that lets site owners state their AI data-usage policies in a clean, standardized format.
Leaving your crawler directives to chance can hurt you in two very different ways. If you’re a publisher or creator, failing to block specific scrapers means your copyrighted material might be ingested without credit or compensation. Conversely, if you want your business to be cited by modern AI assistants, an overly strict bot-protection setup might make you invisible to the AI-driven future of search.
Don’t wait until your content is already swallowed by an LLM dataset. See exactly what the bots see — no signup required, completely free.
Note: sites heavily guarded by Cloudflare or strict web application firewalls may return partial results, as these security layers often block our checker just like they block the bots themselves.