Semafor

semafor.com · United States · measured 9 Oct 2026

Google Search

Allowed

Google-Extended

Allowed

OpenAI

Blocked

Anthropic

Allowed

Perplexity

Allowed

robots.txt

As read on 9 Oct 2026, 233 bytes. 1 rule group name AI crawlers; shown first.

User-agent: GPTBot
Disallow: /
Sitemap: https://www.semafor.com/sitemap.xml
Sitemap: https://www.semafor.com/sitemap-news.xml

User-agent: *
Allow: /
Disallow: /newsletter/
Disallow: /sponsored/
Disallow: /privacy.html
Disallow: /s/

History

Changes recorded since June 2026.

No change recorded.

In context

United States

63.2% block at least one AI crawler

Behind Cloudflare