nw.de · Germany · Zeitungsverlag NEUE WESTFÄLISCHE GmbH & Co. KG · measured 8 Oct 2026
As read on 8 Oct 2026, 4,026 bytes. 4 rule groups name AI crawlers; shown first.
User-agent: GPTBot User-agent: ChatGPT-User User-agent: OAI-SearchBot User-agent: CCBot User-agent: Google-Extended User-agent: ClaudeBot User-agent: Claude-Web User-agent: anthropic-ai User-agent: cohere-ai User-agent: cohere-training-data-crawler User-agent: Meta-ExternalAgent User-agent: FacebookBot User-agent: Applebot-Extended User-agent: YouBot User-agent: Bytespider User-agent: Webzio-Extended User-agent: Diffbot User-agent: PerplexityBot User-agent: DeepSeekBot User-agent: DeepSeek Disallow: / User-agent: AhrefsBot User-agent: SemrushBot User-agent: DataForSeoBot User-agent: MJ12bot User-agent: BLEXBot User-agent: dotbot User-agent: SEOkicks-Robot User-agent: xovi User-agent: SEODAT User-agent: SEOENGBot User-agent: rogerbot User-agent: backlink-check.de User-agent: BacklinkCrawler User-agent: LinkWalker User-agent: LinkextractorPro User-agent: Openbot User-agent: ExtractorPro User-agent: Fasterfox User-agent: magpie-crawler User-agent: SirdataBot User-agent: LCC User-agent: TurnitinBot User-agent: Timpibot User-agent: sentibot User-agent: searchpreview User-agent: True_Robot User-agent: URL Control User-agent: URL_Spider_Pro User-agent: ImagesiftBot Disallow: / User-agent: Meltwater User-agent: NewsNow User-agent: um-IC User-agent: ia_archiver User-agent: Baiduspider User-agent: Baiduspider-image User-agent: Yeti User-agent: omgili User-agent: omgilibot Disallow: / User-agent: Amazonbot Disallow: /20* Sitemap: https://www.nw.de/sitemap_nw_index.xml.gz Sitemap: https://www.nw.de/sitemap_nw_index_news.xml.gz Sitemap: https://www.nw.de/_retresco/sitemap/index.xml Sitemap: https://www.nw.de/_events/sitemap/sitemap_event.xml Sitemap: https://www.nw.de/_apps/nw/cms-api/evocms/modules/evo_cms_sitemap.php Sitemap: https://www.nw.de/sitemap_author.xml User-agent: * Allow: / Disallow: /_em_cms/ Disallow: /cms7/ Disallow: /service/abo_angebote/webabo_checkout Disallow: /tagsuche Disallow: /suche Disallow: /frage/ Disallow: /profil/ Disallow: /microsites/ Disallow: /microsites/mediabox/ Disallow: /anzeigen/native_ads/ Disallow: /anzeigen/native_ads_oms/ Disallow: /nachrichten/wirtschaft/wirtschaftsnewsletter/ Allow: /_em_cms/globals/csslibs.php Allow: /_em_cms/globals/jslibs.php Allow: /_em_cms/globals/acon.php
Changes recorded since June 2026.
No change recorded.