Of 322,595 live Australian websites, only 10,981 (3.4%) block an AI crawler — 14th most open of 43 countries. But Australian newsrooms are among the most defensive on Earth: 25% block AI.
Analysis of 41,818,678 live, content-validated websites worldwide · snapshot 2026-07-13 · by Piperic
Australia's general web is open and unusually decisive: of 322,595 live sites, just 3.4% block an AI crawler — 14th most open of 43 countries — and a striking 14.7% have consciously opted in with a policy file, one of the highest rates in the world (14.99% publish llms.txt). Australians who make a call tend to make it clearly.
Except in the newsroom. Australian news sites block AI at 25.1% — the second-highest press figure in this entire report, behind only South Africa. After years of landmark battles with Google and Meta over news payment, Australian publishers are the least willing anywhere to let AI models read their work for free. The gap between the press (25%) and the national average (3.4%) is more than sevenfold.
Where Australia blocks, it does so reasonably cleanly: only 6.0% of GPTBot-blockers accidentally shut out ChatGPT Search too — near the global norm. And with almost 110,000 Australian shops open to AI crawlers, the country's retail web is already fully visible to AI shopping assistants.
Share of sites blocking at least one AI crawler — lower = more open. Australia ranks #14 of 44 countries analysed.
Neighbours shown for context. Full ranking in the data download.
Every website falls into one of four groups, based on what it blocks and what it declares:
9,857 Australia websites block GPTBot to keep their content out of AI training. But 593 of them — 6.0% — ALSO block OAI-SearchBot, most likely by accident. Those sites have removed themselves from ChatGPT Search results entirely: protected from training, but invisible where their customers now search.
GPTBot (training) and OAI-SearchBot (ChatGPT Search) are independent robots.txt tokens — blocking one does not block the other.
Share of Australia sites blocking each crawler in robots.txt.
Bytespider is ByteDance's (TikTok) crawler — it ignores robots.txt more often than the majors, yet it is among the most-blocked.
AI-blocking and llms.txt adoption across Australia's biggest website categories.
| Sector | Sites | Blocks AI | llms.txt |
|---|---|---|---|
| 📰 News & media | 741 | 25.1% | 3.37% |
| business and finance | 73,143 | 3.4% | 9.09% |
| home and garden | 31,575 | 2.87% | 13.51% |
| technology and computing | 19,005 | 4.23% | 8.55% |
| medical health | 17,325 | 3.65% | 11.61% |
| personal finance | 16,564 | 3.71% | 8.47% |
| automotive | 14,100 | 4.19% | 14.28% |
| style and fashion | 13,781 | 1.97% | 54.26% |
| sports | 12,274 | 4.03% | 19.08% |
| hobbies and interests | 11,690 | 3.59% | 17.05% |
| real estate | 11,588 | 3.37% | 10.36% |
| shopping | 11,119 | 3.58% | 42.09% |
Australia news sites block AI 7.4× more often than the national average (25.1% vs 3.4%) — publishers are the most defensive segment of the web.
107,183 Australia online shops are open to AI crawlers (97.5% of 109,954). Their product content can already be read, compared and recommended by AI assistants — whether they know it or not.
Share of sites blocking at least one AI bot, by platform:
This is the baseline edition (2026-07-13). Piperic re-measures continuously; the next edition will show how fast Australia's web is closing — or opening.
Free, no signup — see your site the way AI crawlers do:
How to cite: “According to Piperic's AI Access Report (2026-07-13), based on 322,595 live Australia websites…” — link to this page.
Download the country dataset (JSON) Global rankings dataset
Sample: 322,595 live, content-validated websites (Australia, ccTLD-based assignment), out of 41,818,678 live domains analysed worldwide. “Live” = HTTP 200 and not parked. “Blocks AI” = the site's robots.txt disallows at least one of 14 known AI crawlers. llms.txt/ai.txt = the file exists at the site root. Snapshot: 2026-07-13 (frozen — numbers do not move with the live crawl). Full-web note: these figures cover the ENTIRE live web including the long tail; industry headlines like “25% of top sites block GPTBot” measure only large publishers — both are true, they measure different things. Country assignment via ccTLD under-counts .com/.org sites; treat US figures as indicative.
Press contact: press@piperic.com · Data: Piperic Business Intelligence
Every number on this page is reproducible from the public dataset above.