CrawlerPulse
CrawlerPulse / Europe / Germany / news.de

News.de

news.de · Germany · MM New Media GmbH · measured 8 Oct 2026

Google Search
Partial
Google-Extended
Partial
OpenAI
Partial
Anthropic
Partial
Perplexity
Partial
Meta
Partial
Common Crawl
Partial
ByteDance
Partial
Apple
Partial
Amazon
Partial

robots.txt

As read on 8 Oct 2026, 847 bytes. No rule names an AI crawler.

Sitemap: https://www.news.de/resources/xml/metasitemap.xml
Sitemap: https://www.news.de/googlesitemap/

User-agent: *
Disallow: /print/
Disallow: /search/
Disallow: /article/
Disallow: /index/
Disallow: /newsfrontend/video/
Disallow: /newsfrontend/imagegallery/
Disallow: /newsfrontend/flash/
Disallow: /newsfrontend/article/
Disallow: /newsfrontend/search/
Disallow: /image/
Disallow: /newsclub/
Disallow: /landing/
Disallow: /textgallery/
Disallow: /resources/
Disallow: /track.php
Disallow: /module/cms/
Allow: /resources/xml/
Allow: /images/
Allow: /resources/images/
Allow: /resources/thumbs/

User-agent: Googlebot-News
Disallow: /article/
Disallow: /morgenattacke/
Disallow: /special/
Disallow: /fotostrecke/
Disallow: /video/

User-agent: Yahoo!-AdCrawler
Allow: /search/
Crawl-delay: 0.5

User-agent: Mediapartners-Google
Allow: /search/

History

Changes recorded since June 2026.

No change recorded.

In context

Germany
71.7% block at least one AI crawler
Behind Cloudflare
No