CrawlerPulse
CrawlerPulse / Europe / Germany / berliner-zeitung.de

Berliner Zeitung

berliner-zeitung.de · Germany · Berliner Verlag GmbH · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 1,039 bytes. No rule names an AI crawler.

User-agent: Googlebot-News
Allow: /

User-agent: EchoboxBot
Allow: /

User-agent: Mozilla/5.0 (compatible; OGDWCtxCrawler)
Allow: /

User-agent: Mediapartners-Google
Allow: /

User-agent: *
Disallow: /suche?q=
Disallow: /jos-data
Disallow: /api/
Disallow: /*.woff2$
Disallow: /*.woff$
Disallow: /*.ttf$
Disallow: /cdn-cgi/
Disallow: /en-US
Disallow: /tests
Disallow: /status
Disallow: /session/
Disallow: /search
Disallow: /article-metadata
Disallow: /articles-from-category
Disallow: /designs/
Disallow: /ajax/
Disallow: /interaktiv/
Disallow: /tracking/
Disallow: /tagcloud/
Crawl-delay: 10
Allow: /.well-known/
Sitemap: https://www.berliner-zeitung.de/sitemap.xml
Sitemap: https://www.berliner-zeitung.de/news-sitemap.xml
Sitemap: https://www.berliner-zeitung.de/sitemap.articles.xml
Sitemap: https://www.berliner-zeitung.de/sitemap.authors.xml
Sitemap: https://www.berliner-zeitung.de/sitemap.pages.xml
Sitemap: https://www.berliner-zeitung.de/sitemap.static-pages.xml
Sitemap: https://www.berliner-zeitung.de/sitemap.topics.xml

History

Changes recorded since June 2026.

No change recorded.

In context

Germany
71.7% block at least one AI crawler
Behind Cloudflare
Yes
Same group