Category
Primary
web-development · embed-ft 0.7
Tree path
Technology & Computing › Computing › Internet › Web Development
Group (tier-1)
AI readiness
Compliance (GEO / GDPR)
TLD
Overview
Title
Robots.txt Checker – See what bots are allowed to crawl your site
Description
Enter a website URL and see what search and AI crawlers are allowed or blocked by its robots.txt file. Get a simple robots.txt report for your site.
Final URL
Language
en (html)
Scanned at
2026-06-26 14:35:05