CrawlerPulse
CrawlerPulse / Europe / Germany / shz.de

SHZ

shz.de · Germany · sh:z Schleswig-Holsteinischer Zeitungsverlag GmbH & Co. KG · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Blocked
OpenAI
Blocked
Anthropic
Blocked
Perplexity
Blocked
Meta
Blocked
Common Crawl
Blocked
ByteDance
Blocked
Apple
Blocked
Amazon
Blocked

robots.txt

As read on 8 Oct 2026, 2,610 bytes. 1 rule group name AI crawlers; shown first.

User-agent: AI2Bot
User-agent: Amazonbot
User-agent: anthropic-ai
User-agent: Applebot-Extended
User-agent: Bytespider
User-agent: CCBot
User-agent: ChatGPT-User
User-agent: Claude-SearchBot
User-agent: Claude-User
User-agent: Claude-Web
User-agent: ClaudeBot
User-agent: CloudVertexBot
User-agent: cohere-ai
User-agent: cohere-training-data-crawler
User-agent: DeepSeek
User-agent: DeepSeekBot
User-agent: Devin
User-agent: Diffbot
User-agent: DuckAssistBot
User-agent: FacebookBot
User-agent: GPTBot
User-agent: Google-Extended
User-agent: ImagesiftBot
User-agent: Meta-ExternalAgent
User-agent: MistralAI-User
User-agent: OAI-SearchBot
User-agent: omgili
User-agent: omgilibot
User-agent: PerplexityBot
User-agent: PetalBot
User-agent: YouBot
Disallow: /

User-Agent: Mediapartners-Google
Disallow:

User-Agent: *
Disallow: /pagebuilding
Disallow: /pagebuilding/
Disallow: /new-articles
Disallow: /new-articles/
Disallow: /suche
Disallow: /*?q=*
Disallow: /files/
Disallow: /cre-1.0/tracking/
Disallow: /*MID=
Disallow: /*utm_source=
Disallow: /*source_vicksweb=
Disallow: /*editor=
Disallow: /*Editor=
Disallow: /*itm_content=
Disallow: /*page=
Disallow: /*dsplz=
Disallow: /*itm_medium=
Disallow: /*r=
Disallow: /*location_key=
Disallow: /*fbclid=
Disallow: /*womort=
Disallow: /*ref=
Disallow: /*artId=
Disallow: /*cuid=
Disallow: /*itm_campaign=
Disallow: /*sfnsn=
Disallow: /*RUBRIKID=
Disallow: /*article-scroller=
Disallow: /*user_dimbildergalerie_pi1=
Disallow: /*utm_medium=
Disallow: /*s=
Disallow: /*k=
Disallow: /*noredirect=
Disallow: /*utm_content=
Disallow: /*token=
Disallow: /*utm_campaign=
Disallow: /*dslayout=
Disallow: /*dslayoutpost=
Disallow: /*cx_testId=
Disallow: /*cx_testVariant=
Disallow: /*cx_artPos=
Disallow: /*cx_experienceId=
Disallow: /*cx_experienceActionId=
Disallow: /*previewAll=
Disallow: /*dicbo=
Disallow: /*author=
Disallow: /*q=
Disallow: /*rel=
Disallow: /*categoryAlias=
Disallow: /*set=
Disallow: /*alternatehome=
Disallow: /*x-client
Disallow: /*suchbegriff=
Disallow: /login
Disallow: /widget/
Sitemap: https://www.shz.de/sitemap.xml
Sitemap: https://www.shz.de/sitemap/googleNewsList.xml
Sitemap: https://www.shz.de/sitemap/artikel/sitemap-current.xml

History

Changes recorded since June 2026.

No change recorded.

In context

Germany
71.7% block at least one AI crawler
Behind Cloudflare
No