CrawlerPulse
CrawlerPulse / Europe / Germany / focus.de

Focus

focus.de · Germany · Burda Forward GmbH · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Blocked
Anthropic
Blocked
Perplexity
Allowed
Meta
Blocked
Common Crawl
Blocked
ByteDance
Blocked
Apple
Allowed
Amazon
Blocked

robots.txt

As read on 8 Oct 2026, 1,648 bytes. 10 rule groups name AI crawlers; shown first.

User-agent: Amazonbot
Disallow: /

User-agent: Bytespider
Disallow: /

User-agent: CCBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: FacebookBot
Disallow: /

User-agent: Google-CloudVertexBot
Disallow: /

User-agent: GPTBot
Disallow: /

User-agent: meta-externalagent
Disallow: /

User-agent: PetalBot
Disallow: /

User-agent: Timpibot
Disallow: /

User-agent: *
Disallow: /test/
Disallow: /*/test/
Disallow: /service/archiv/
Disallow: /incoming/
Allow: /incoming/*.ece$
Allow: /incoming/archiv/*_id_*.html$
Disallow: /apple-app-site-association
Disallow: /src/js/spmsg*

User-agent: Mediapartners-Google
Allow: /

User-agent: MegaIndex.ru
Disallow: /

User-agent: trendkite-akashic-crawler
Disallow: /

User-agent: Anchor Browser
Disallow: /

User-agent: Novellum
Disallow: /

User-agent: ProRataInc
Disallow: /

User-agent: FriendlyCrawler
Disallow: /

History

Changes recorded since June 2026.

No change recorded.

In context

Germany
71.7% block at least one AI crawler
Behind Cloudflare
No
Same group