CrawlerPulse
CrawlerPulse / Crawlers / ClaudeBot

ClaudeBot

Anthropic. Collects public web content that may be used to train Anthropic's Claude models; search and user-directed retrieval use other bots. Anthropic's documentation.

22.8%

of 12,650 measured news outlets block ClaudeBot.

Each square ≈ 127 outlets

By continent

Share of each continent's measured outlets blocking ClaudeBot.

Africa1,448
8%
Asia2,045
12%
Europe4,145
31%
Oceania341
24%

By country

Countries with 50 or more outlets measured, from the most blocking to the least.

Finland153
70%
Norway143
65%
Germany244
61%
Sweden178
56%
Ireland84
45%
Italy222
43%
Canada199
39%
Australia252
28%
France397
28%
Spain399
27%
Denmark63
27%
Japan307
26%
Austria70
24%
Poland165
19%
Czechia71
16%
Indonesia102
15%
India249
14%
Portugal104
13%
Brazil283
10%
Ghana77
9%
Turkey101
9%
Greece116
9%
Peru78
8%
Mexico403
7%
China71
6%
Iran54
6%
Cyprus61
5%
Ukraine145
5%
Argentina1,093
5%
Nigeria226
4%
Romania126
3%
Russia158
3%
Kenya106
3%
Chile149
3%
Serbia79
3%

Largest outlets that block ClaudeBot

By public reach. One cell per crawler.

Google SearchGoogle-ExtendedOpenAIAnthropicPerplexityMetaCommon CrawlByteDanceAppleAmazonof 9
endpoints.news · United States
5
nytimes.com · United States
8
t-online.de · Germany
7
medium.com · United States
6
theguardian.com · United Kingdom
7
cnn.com · United States
8
forbes.com · United States
8
bbc.com · Brazil
9
bbc.co.uk · United Kingdom
9
reuters.com · United States
9
washingtonpost.com · United States
7
bloomberg.com · United States
9
wsj.com · United States
7
businessinsider.com · United States
3
npr.org · United States
7
uol.com.br · Brazil
8
wired.com · United States
8
ft.com · United Kingdom
7
usatoday.com · United States
9
techcrunch.com · United States
6
Google SearchGoogle-ExtendedOpenAIAnthropicPerplexityMetaCommon CrawlByteDanceAppleAmazonof 9
elpais.com · Spain
5
lemonde.fr · France
6
dailymail.co.uk · United Kingdom
9
cnet.com · United States
8
apnews.com · United States
6
people.com · United States
8
lefigaro.fr · France
5
theverge.com · United States
7
nbcnews.com · United States
9
focus.de · Germany
6
latimes.com · United States
3
theatlantic.com · United States
8
ouest-france.fr · France
9
huffpost.com · United States
9
as.com · Spain
5
bfmtv.com · France
9
aftonbladet.se · Sweden
8
nypost.com · United States
8
economist.com · United Kingdom
7
elconfidencial.com · Spain
9
BlockedAllowed

Largest outlets that let it in

By public reach.

Google SearchGoogle-ExtendedOpenAIAnthropicPerplexityMetaCommon CrawlByteDanceAppleAmazonof 9
facebook.com · Australia
3
globo.com · Brazil
3
espn.com · United States
4
indiatimes.com · India
1
foxnews.com · United States
0
time.com · United States
0
telegraph.co.uk · United Kingdom
1
cbsnews.com · United States
1
independent.co.uk · United Kingdom
0
bild.de · Germany
5
Google SearchGoogle-ExtendedOpenAIAnthropicPerplexityMetaCommon CrawlByteDanceAppleAmazonof 9
nikkei.com · Japan
0
welt.de · Germany
5
ameblo.jp · Japan
0
onet.pl · Poland
3
repubblica.it · Italy
6
elmundo.es · Spain
4
infobae.com · Argentina
0
n-tv.de · Germany
3
pbs.org · United States
4