ndr.de · Germany · measured 9 Oct 2026
As read on 9 Oct 2026, 2,495 bytes. 12 rule groups name AI crawlers; shown first.
User-agent: applebot-extended Disallow: / User-agent: bytespider Disallow: / User-agent: ccbot Disallow: / User-agent: claudebot Disallow: / User-agent: google-extended Disallow: / User-agent: deepseekbot Disallow: / User-agent: diffbot Disallow: / User-agent: facebookbot Disallow: / User-agent: meta-externalagent Disallow: / User-agent: gptbot Disallow: / User-agent: omgili Disallow: / User-agent: omgilibot Disallow: / Sitemap: https://www.ndr.de/news-102~news.xml Sitemap: https://www.ndr.de/index~sitemap.xml User-agent: * Disallow: /n-joy/appmedien Disallow: /*ut*.xml Disallow: /*podcast5654.xml Disallow: /*.json Disallow: /ratgeber/*?filter= Disallow: /*~ardplayer.html Disallow: /*.m3u8 Disallow: /public/teletext/ User-agent: cohere-training-data-crawler Disallow: / User-agent: deepseek Disallow: / User-agent: deepseek-crawler Disallow: / User-agent: grokbot Disallow: / User-agent: img2dataset Disallow: / User-agent: laiondownloader Disallow: / User-agent: mistralai-crawler Disallow: / User-agent: pangubot Disallow: /
Changes recorded since June 2026.
No change recorded.