Category & trade
Primary
technology-and-computing · embed-ft 0.33
Tree path
Technology & Computing
Group (tier-1)
Also (top-3)
web-development0.29
Tech stack
AI readiness
AI-Readiness score
28 / 100 · not-ready · see similar
AI training policy
allowed
AI-bot protection
AI files
Compliance (GEO / GDPR)
TLD
Overview
Title
robots.txt Tools & Resources
Description
Tools and guides to manage and optimize your robots.txt file. Created by ex-Google engineer Fili.
Final URL
Language
en (content)
Scanned at
2026-07-10 00:49:57