Of 755,838 live UK websites, 31,716 (4.2%) block at least one AI crawler — above the EU average and 26th of 43 countries. But the UK also has one of the highest rates of deliberate openness in the world.
Analysis of 41,818,678 live, content-validated websites worldwide · snapshot 2026-07-13 · by Piperic
The UK runs one of the largest and most self-aware webs in the world. Of 755,838 live British websites, 4.2% block an AI crawler — above the EU average (3.56%) and 26th of 43 countries — yet Britain also leads almost everyone on the opposite measure: 9.5% have consciously opted in with an llms.txt or ai.txt, and 9.71% publish llms.txt, among the highest adoption rates recorded here.
British newsrooms buck the European trend. Where German (9.0%), Czech (14.2%) and Australian (25%) publishers barricade themselves, UK news sites block AI at just 3.57% — below the national average. The British press, for now, is choosing visibility. The defensive crown instead goes to attractions (7.23%) and shopping (6.41%).
The hidden cost sits in the firewalls: 9.3% of UK GPTBot-blockers — 2,639 sites — also blocked OAI-SearchBot, most likely by accident, erasing themselves from ChatGPT Search. And with 175,000 British shops open to AI crawlers, the UK's product web is already being read, compared and recommended by AI assistants at scale.
Share of sites blocking at least one AI crawler — lower = more open. United Kingdom ranks #26 of 44 countries analysed.
Neighbours shown for context. Full ranking in the data download.
Every website falls into one of four groups, based on what it blocks and what it declares:
28,499 United Kingdom websites block GPTBot to keep their content out of AI training. But 2,639 of them — 9.3% — ALSO block OAI-SearchBot, most likely by accident. Those sites have removed themselves from ChatGPT Search results entirely: protected from training, but invisible where their customers now search.
GPTBot (training) and OAI-SearchBot (ChatGPT Search) are independent robots.txt tokens — blocking one does not block the other.
Share of United Kingdom sites blocking each crawler in robots.txt.
Bytespider is ByteDance's (TikTok) crawler — it ignores robots.txt more often than the majors, yet it is among the most-blocked.
AI-blocking and llms.txt adoption across United Kingdom's biggest website categories.
| Sector | Sites | Blocks AI | llms.txt |
|---|---|---|---|
| 📰 News & media | 8,366 | 3.57% | 1.24% |
| business and finance | 137,055 | 3.4% | 7.77% |
| home and garden | 71,009 | 3.35% | 11.37% |
| technology and computing | 57,799 | 4.09% | 5.55% |
| attractions | 41,882 | 7.23% | 3.22% |
| hobbies and interests | 39,114 | 4.65% | 10.71% |
| automotive | 32,211 | 4.59% | 9.24% |
| sports | 31,815 | 6.0% | 9.63% |
| medical health | 28,846 | 3.14% | 8.26% |
| style and fashion | 26,611 | 3.37% | 36.26% |
| personal finance | 24,832 | 3.32% | 6.18% |
| real estate | 23,724 | 3.7% | 5.45% |
United Kingdom news sites block AI 0.8× more often than the national average (3.57% vs 4.2%) — publishers are the most defensive segment of the web.
168,191 United Kingdom online shops are open to AI crawlers (96.2% of 174,926). Their product content can already be read, compared and recommended by AI assistants — whether they know it or not.
Share of sites blocking at least one AI bot, by platform:
This is the baseline edition (2026-07-13). Piperic re-measures continuously; the next edition will show how fast United Kingdom's web is closing — or opening.
Free, no signup — see your site the way AI crawlers do:
How to cite: “According to Piperic's AI Access Report (2026-07-13), based on 755,838 live United Kingdom websites…” — link to this page.
Download the country dataset (JSON) Global rankings dataset
Sample: 755,838 live, content-validated websites (United Kingdom, ccTLD-based assignment), out of 41,818,678 live domains analysed worldwide. “Live” = HTTP 200 and not parked. “Blocks AI” = the site's robots.txt disallows at least one of 14 known AI crawlers. llms.txt/ai.txt = the file exists at the site root. Snapshot: 2026-07-13 (frozen — numbers do not move with the live crawl). Full-web note: these figures cover the ENTIRE live web including the long tail; industry headlines like “25% of top sites block GPTBot” measure only large publishers — both are true, they measure different things. Country assignment via ccTLD under-counts .com/.org sites; treat US figures as indicative.
Press contact: press@piperic.com · Data: Piperic Business Intelligence
Every number on this page is reproducible from the public dataset above.