Crawlson · AI crawler
Crawlson
Search indexRespects robots.txtSearch Engine Crawler
Crawlson is a search engine crawler for the crawlson.com search engine.
Official documentation ↗
🛰️ We haven’t observed Crawlson crawling piperic.com yet. This page updates automatically when it does.
Block Crawlson in robots.txt
Add this to your site’s /robots.txt:
User-agent: Crawlson
Disallow: /
← All AI bots