freiepresse.de · Germany · Medien Union GmbH · measured 8 Oct 2026
As read on 8 Oct 2026, 3,261 bytes. 11 rule groups name AI crawlers; shown first.
User-agent: GPTBot Disallow: / User-agent: OAI-SearchBot Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Claude-Web Disallow: / User-agent: Google-Extended Disallow: / User-agent: PerplexityBot Disallow: / User-agent: DeepSeekBot Disallow: / User-agent: Bytespider Disallow: / User-agent: Amazonbot Disallow: / User-agent: CCBot Disallow: / Sitemap: https://www.freiepresse.de/sitemaps/articles_news.xml Sitemap: https://www.freiepresse.de/sitemap.xml User-agent: * Disallow: /testsite/ Disallow: /testseite/ Disallow: /service/servicepartner/ Disallow: /FILES/SSP/ Disallow: /erfahrungen/go/ Disallow: /erfahrungen/wp-admin/ Disallow: /suche$ Disallow: /suche?q=* Disallow: /showmappe Disallow: /showmappe/ Disallow: /ct Disallow: /pm Disallow: /TRASH/* User-agent: backlink-check.de Disallow: / User-agent: BacklinkCrawler Disallow: / User-agent: Bloodhound Disallow: / User-agent: cydralspider Disallow: / User-agent: downloadexpress Disallow: / User-agent: ExtractorPro Disallow: / User-agent: Fasterfox Disallow: / User-agent: gammaSpider Disallow: / User-agent: LinkextractorPro Disallow: / User-agent: LinkWalker Disallow: / User-agent: Meltwater Disallow: / User-agent: MJ12bot Disallow: / User-agent: NewsNow Disallow: / User-agent: Node/simplecrawler Disallow: / User-agent: Node/simplecrawler 0.7.0 (git+https://github.com/cgiffard/node-simplecrawler.git) Disallow: / User-agent: ObjectsSearch Disallow: / User-agent: Openbot Disallow: / User-agent: Pimptrain Disallow: / User-agent: Raven Disallow: / User-agent: rogerbot Disallow: / User-agent: searchpreview Disallow: / User-agent: simplecrawler Disallow: / User-agent: SEODAT Disallow: / User-agent: SEOENGBot Disallow: / User-agent: SEOkicks-Robot Disallow: / User-agent: True_Robot Disallow: / User-agent: URL Control Disallow: / User-agent: URL_Spider_Pro Disallow: / User-agent: wapspider Disallow: / User-agent: WebZinger Disallow: / User-agent: xovi Disallow: / User-agent: Claude-SearchBot Disallow: / User-agent: Claude-Code Disallow: / User-agent: Claude-User Disallow: / User-agent: Perplexity-User Disallow: / User-agent: DeepSeek Disallow: / User-agent: ByteDanceSpider Disallow: / User-agent: Baiduspider Disallow: / User-agent: ia_archiver Disallow: / User-agent: ia_archiver-web.archive.org Disallow: / User-agent: archive.org_bot Disallow: / User-agent: archive.today Disallow: / User-agent: archive.is Disallow: / User-agent: archive.ph Disallow: / User-agent: TencentBot Disallow: / User-agent: TencenTraveler Disallow: /
Changes recorded since June 2026.
No change recorded.