Of 176,930 live Indian websites, 8,666 (4.9%) block an AI crawler — 34th of 43 countries. Yet one sector stands apart: nearly a quarter of Indian attractions and tourism sites block AI.
Analysis of 41,818,678 live, content-validated websites worldwide · snapshot 2026-07-13 · by Piperic
India runs one of the largest webs in this survey. Of 176,930 live Indian sites, 4.9% block an AI crawler — 34th of 43 countries, above the global norm — while 7.9% have consciously opted in with a policy file. The overwhelming majority, 87%, sit in silence, neither blocking nor declaring.
One sector breaks the pattern dramatically: attractions and tourism block AI at 23.45% — nearly five times the national average and one of the highest single-sector figures anywhere in this report. In a country whose travel and heritage sites draw global attention, they are the least visible to the AI assistants travellers increasingly ask. Careers sites follow far behind at 5.57%.
Where India blocks, it blocks with unusual precision: just 3.2% of GPTBot-blockers accidentally lost ChatGPT Search — among the lowest rates in the world, roughly half the global 6.1%. Indian administrators clearly distinguish training bots from search bots.
Share of sites blocking at least one AI crawler — lower = more open. India ranks #34 of 44 countries analysed.
Neighbours shown for context. Full ranking in the data download.
Every website falls into one of four groups, based on what it blocks and what it declares:
8,022 India websites block GPTBot to keep their content out of AI training. But 259 of them — 3.2% — ALSO block OAI-SearchBot, most likely by accident. Those sites have removed themselves from ChatGPT Search results entirely: protected from training, but invisible where their customers now search.
GPTBot (training) and OAI-SearchBot (ChatGPT Search) are independent robots.txt tokens — blocking one does not block the other.
Share of India sites blocking each crawler in robots.txt.
Bytespider is ByteDance's (TikTok) crawler — it ignores robots.txt more often than the majors, yet it is among the most-blocked.
AI-blocking and llms.txt adoption across India's biggest website categories.
| Sector | Sites | Blocks AI | llms.txt |
|---|---|---|---|
| 📰 News & media | 2,680 | 7.8% | 5.97% |
| business and finance | 53,366 | 3.43% | 4.38% |
| technology and computing | 23,242 | 4.87% | 4.8% |
| education | 13,412 | 4.73% | 3.62% |
| style and fashion | 7,409 | 2.93% | 47.89% |
| home and garden | 6,996 | 4.06% | 10.61% |
| shopping | 6,738 | 4.1% | 31.55% |
| personal finance | 6,296 | 4.45% | 3.95% |
| travel | 6,097 | 4.22% | 3.94% |
| medical health | 5,845 | 5.08% | 6.62% |
| attractions | 5,607 | 23.45% | 2.46% |
| careers | 3,663 | 5.57% | 4.45% |
India news sites block AI 1.6× more often than the national average (7.8% vs 4.9%) — publishers are the most defensive segment of the web.
29,842 India online shops are open to AI crawlers (96.3% of 30,982). Their product content can already be read, compared and recommended by AI assistants — whether they know it or not.
Share of sites blocking at least one AI bot, by platform:
This is the baseline edition (2026-07-13). Piperic re-measures continuously; the next edition will show how fast India's web is closing — or opening.
Free, no signup — see your site the way AI crawlers do:
How to cite: “According to Piperic's AI Access Report (2026-07-13), based on 176,930 live India websites…” — link to this page.
Download the country dataset (JSON) Global rankings dataset
Sample: 176,930 live, content-validated websites (India, ccTLD-based assignment), out of 41,818,678 live domains analysed worldwide. “Live” = HTTP 200 and not parked. “Blocks AI” = the site's robots.txt disallows at least one of 14 known AI crawlers. llms.txt/ai.txt = the file exists at the site root. Snapshot: 2026-07-13 (frozen — numbers do not move with the live crawl). Full-web note: these figures cover the ENTIRE live web including the long tail; industry headlines like “25% of top sites block GPTBot” measure only large publishers — both are true, they measure different things. Country assignment via ccTLD under-counts .com/.org sites; treat US figures as indicative.
Press contact: press@piperic.com · Data: Piperic Business Intelligence
Every number on this page is reproducible from the public dataset above.