CrawlerPulse
CrawlerPulse / Europe / Germany / siegener-zeitung.de

Siegener Zeitung

siegener-zeitung.de · Germany · Verlagsgesellschaft Madsack GmbH & Co. KG · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Blocked
OpenAI
Blocked
Anthropic
Blocked
Perplexity
Blocked
Meta
Allowed
Common Crawl
Blocked
ByteDance
Blocked
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 1,215 bytes. 7 rule groups name AI crawlers; shown first.

User-agent: GPTBot
Disallow: /

User-agent: CCBot
Disallow: /

User-agent: ChatGPT-User
Disallow: /

User-agent: Bytespider
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: PerplexityBot
Disallow: /

User-agent: Google-Extended
Disallow: /

User-agent: *
Disallow: /mandanten/
Disallow: /mediabox/
Disallow: /politik/politik-extern/
Disallow: /wirtschaft/wirtschaft-extern
Disallow: /suche/
Disallow: /ellipsis-preview/
Disallow: /zeitung/
Disallow: /metaseiten/
Disallow: /var/storage
Disallow: /var/storage/*
Disallow: /pf/
Allow: /pf/resources/favicons/*
Allow: /pf/dist/components/combinations/*
Allow: /ueber-uns/
Disallow: /bundles/
Disallow: /cms/
Disallow: /security/
Disallow: /newsletter/abmeldung/
Disallow: /angebot/
Disallow: /veranstaltungskalender/

History

Changes recorded since June 2026.

No change recorded.

In context