CrawlerPulse
CrawlerPulse / Crawlers / Googlebot

Googlebot

Google. The crawler of Google Search, shown here for comparison: blocking it removes a site from Google. Google's documentation.

1.4%

of 12,650 measured news outlets block Googlebot.

Each square ≈ 127 outlets

By continent

Share of each continent's measured outlets blocking Googlebot.

Africa1,448
2%
Asia2,045
2%
Europe4,145
1%
Oceania341
1%

By country

Countries with 50 or more outlets measured, from the most blocking to the least.

Largest outlets that block Googlebot

By public reach. One cell per crawler.

Google SearchGoogle-ExtendedOpenAIAnthropicPerplexityMetaCommon CrawlByteDanceAppleAmazonof 9
telegraph.co.uk · United Kingdom
1
arstechnica.com · United States
9
cctv.com · China
9
nrk.no · Norway
9
srf.ch · Switzerland
9
lexisnexis.com · United States
9
letelegramme.fr · France
9
cnews.fr · France
9
livehindustan.com · India
9
noticiasaominuto.com · Portugal
9
nikkeibp.co.jp · Japan
9
joins.com · South Korea
9
rts.ch · Switzerland
9
vol.at · Austria
9
apa.at · Austria
9
wcvb.com · United States
9
ntnews.com · India
9
kcra.com · United States
9
caixin.com · China
9
catholicnewsagency.com · United States
9
Google SearchGoogle-ExtendedOpenAIAnthropicPerplexityMetaCommon CrawlByteDanceAppleAmazonof 9
on.cc · Hong Kong
9
wmur.com · United States
9
watson.de · Germany
9
wlwt.com · United States
9
kcci.com · United States
9
chita.ru · Russia
9
wtae.com · United States
9
wesh.com · United States
9
wyff4.com · United States
9
todayonline.com · Singapore
9
wgal.com · United States
9
eg.ru · Russia
9
aksam.com.tr · Turkey
9
fomos.kr · South Korea
9
sofoot.com · France
9
ketv.com · United States
9
wdsu.com · United States
9
koco.com · United States
9
vienna.at · Austria
9
kmbc.com · United States
9
BlockedAllowed

Largest outlets that let it in

By public reach.

Google SearchGoogle-ExtendedOpenAIAnthropicPerplexityMetaCommon CrawlByteDanceAppleAmazonof 9
facebook.com · Australia
3
endpoints.news · United States
5
nytimes.com · United States
8
t-online.de · Germany
7
medium.com · United States
6
theguardian.com · United Kingdom
7
cnn.com · United States
8
forbes.com · United States
8
bbc.com · Brazil
9
bbc.co.uk · United Kingdom
9
Google SearchGoogle-ExtendedOpenAIAnthropicPerplexityMetaCommon CrawlByteDanceAppleAmazonof 9
reuters.com · United States
9
washingtonpost.com · United States
7
bloomberg.com · United States
9
wsj.com · United States
7
globo.com · Brazil
3
espn.com · United States
4
businessinsider.com · United States
3
npr.org · United States
7
indiatimes.com · India
1
foxnews.com · United States
0