CrawlerPulse
CrawlerPulse / Crawlers / Applebot-Extended

Applebot-Extended

Apple. Not a separate crawler but a robots.txt token: it lets a site keep its content out of the training of Apple's generative AI models, while still appearing in Apple's search features. Apple's documentation.

18.8%

of 12,650 measured news outlets block Applebot-Extended.

Each square ≈ 127 outlets

By continent

Share of each continent's measured outlets blocking Applebot-Extended.

Africa1,448
7%
Asia2,045
11%
Europe4,145
25%
Oceania341
16%

By country

Countries with 50 or more outlets measured, from the most blocking to the least.

Norway143
59%
Germany244
52%
Italy222
41%
Ireland84
38%
Canada199
37%
Finland153
29%
Denmark63
27%
Japan307
25%
France397
23%
Sweden178
21%
Indonesia102
20%
Poland165
19%
Australia252
19%
Spain399
19%
Austria70
13%
Czechia71
13%
Portugal104
12%
India249
10%
Ghana77
8%
Brazil283
7%
Peru78
6%
China71
6%
Iran54
6%
Greece116
5%
Mexico403
5%
Argentina1,093
4%
Nigeria226
4%
Ukraine145
3%
Cyprus61
3%
Romania126
3%
Russia158
3%
Turkey101
3%
Kenya106
3%
Chile149
3%
Serbia79
3%

Largest outlets that block Applebot-Extended

By public reach. One cell per crawler.

Google SearchGoogle-ExtendedOpenAIAnthropicPerplexityMetaCommon CrawlByteDanceAppleAmazonof 9
nytimes.com · United States
8
t-online.de · Germany
7
medium.com · United States
6
theguardian.com · United Kingdom
7
cnn.com · United States
8
forbes.com · United States
8
bbc.com · Brazil
9
bbc.co.uk · United Kingdom
9
reuters.com · United States
9
washingtonpost.com · United States
7
bloomberg.com · United States
9
npr.org · United States
7
uol.com.br · Brazil
8
wired.com · United States
8
ft.com · United Kingdom
7
usatoday.com · United States
9
telegraph.co.uk · United Kingdom
1
techcrunch.com · United States
6
lemonde.fr · France
6
dailymail.co.uk · United Kingdom
9
Google SearchGoogle-ExtendedOpenAIAnthropicPerplexityMetaCommon CrawlByteDanceAppleAmazonof 9
cnet.com · United States
8
bild.de · Germany
5
welt.de · Germany
5
apnews.com · United States
6
people.com · United States
8
theverge.com · United States
7
nbcnews.com · United States
9
repubblica.it · Italy
6
theatlantic.com · United States
8
ouest-france.fr · France
9
huffpost.com · United States
9
bfmtv.com · France
9
aftonbladet.se · Sweden
8
nypost.com · United States
8
economist.com · United Kingdom
7
elconfidencial.com · Spain
9
theconversation.com · Australia
9
vg.no · Norway
8
gazzetta.it · Italy
7
merkur.de · Germany
8
BlockedAllowed

Largest outlets that let it in

By public reach.

Google SearchGoogle-ExtendedOpenAIAnthropicPerplexityMetaCommon CrawlByteDanceAppleAmazonof 9
facebook.com · Australia
3
endpoints.news · United States
5
wsj.com · United States
7
globo.com · Brazil
3
espn.com · United States
4
businessinsider.com · United States
3
indiatimes.com · India
1
foxnews.com · United States
0
time.com · United States
0
elpais.com · Spain
5
Google SearchGoogle-ExtendedOpenAIAnthropicPerplexityMetaCommon CrawlByteDanceAppleAmazonof 9
cbsnews.com · United States
1
independent.co.uk · United Kingdom
0
nikkei.com · Japan
0
ameblo.jp · Japan
0
lefigaro.fr · France
5
focus.de · Germany
6
latimes.com · United States
3
onet.pl · Poland
3
elmundo.es · Spain
4