AI crawler comparison
Amazonbot vs meta-externalagent
Amazon
Model trainingRespects robots.txt
2,228,896
websites block it · 4.469% of the web · #3
VS
Meta
Model trainingRespects robots.txt
2,127,714
websites block it · 4.266% of the web · #7
The verdict
Amazonbot is blocked by more websites than meta-externalagent (2,228,896 vs 2,127,714).
| Amazonbot | meta-externalagent |
|---|
| Operator | Amazon | Meta |
| Purpose | Model training | Model training |
| Respects robots.txt | Respects robots.txt | Respects robots.txt |
| Category | AI Data Scrapers | AI Data Scrapers |
| Websites blocking it | 2,228,896 | 2,127,714 |
| Share of the web | 4.469% | 4.266% |
| Rank | #3 most-blocked | #7 most-blocked |
| Verify method | ip-range | ip-range |
⚖️ Compare any two AI crawlers
vs
Block both in robots.txt
Add this to your site’s /robots.txt:
User-agent: Amazonbot
User-agent: meta-externalagent
Disallow: /
Other comparisons
← AI Bot Observatory · Full database →