CrawlerPulse
Last updated {{ lastRun }}
{{ r.title }}{{ r.tag }}
{{ r.sub }}
{{ r.meta }}
No matches.

Which news outlets block AI crawlers, measured from their robots.txt.

{{ anyPerc }}%

{{ t.text }}
{{ squareText }}

By crawler

{{ crawlersText }}

{{ c.name }}
{{ c.label }}

By size

{{ sizeText }}

{{ z.perc }}%
{{ z.name }}
{{ z.n }} outlets

By country

Share of outlets blocking Google-Extended. Countries with fewer than five outlets measured are left blank.

Up to 20%
20–50%
Over 50%
No data
Country · outlets
50+ outlets measured
Block
{{ k.name }}{{ k.n }}
{{ k.label }}

Largest outlets

One cell per crawler. Count excludes Google Search.

{{ h }}
of 9
{{ o.name }}
{{ o.domain }} · {{ o.country }}
{{ o.count }}

No outlets match.
Blocked
Allowed
View all outlets

Changes since the previous run

{{ changesText }}

{{ g.count }}
{{ g.title }}
{{ g.sub }}
{{ ch.date }}
{{ ch.name }}
{{ ch.domain }}
{{ ch.country }}

Over time

Outlets blocking OpenAI, one point per run. The series starts in September 2026.

100%
50%
0%
{{ trendLabel }}
{{ lastRun }}Next run: October 2026