CrawlerPulse
CrawlerPulse / Europe / Germany / rtl.de

RTL

rtl.de · Germany · RTL Deutschland GmbH · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Blocked
OpenAI
Blocked
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Blocked
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 1,230 bytes. 3 rule groups name AI crawlers; shown first.

User-agent: GPTBot
Disallow: /
Allow: /vergleiche/
Allow: /casinos/
Allow: /sportwetten/
Allow:/tools/

User-agent: Google-Extended
Disallow: /
Allow: /vergleiche/
Allow: /casinos/
Allow: /sportwetten/
Allow:/tools/

User-agent: CCBot
Disallow: /
Allow: /vergleiche/
Allow: /casinos/
Allow: /sportwetten/
Allow:/tools/
Sitemap: https://www.rtl.de/sitemap-index.xml
Sitemap: https://www.rtl.de/sitemap-news.xml
Sitemap: https://www.rtl.de/sitemap-section.xml
Sitemap: https://www.rtl.de/sitemap-video.xml
Sitemap: https://www.rtl.de/tools/zahnzusatzversicherung/sitemap.xml
Sitemap: https://www.rtl.de/tools/hoerbuecher/sitemap.xml
Sitemap: https://www.rtl.de/tools/gps-tracker/sitemap.xml
Sitemap: https://www.rtl.de/tools/sprachen-lernen/sitemap.xml
Sitemap: https://www.rtl.de/tools/lieferservice/sitemap.xml
Sitemap: https://www.rtl.de/tools/abnehmen/sitemap.xml
Sitemap: https://www.rtl.de/tools/tierversicherung/sitemap.xml
Sitemap: https://www.rtl.de/tools/kochbox/sitemap-0.xml

User-agent: *
Disallow: /archiv/
Disallow: /kreditkarte/infos/
Disallow: /mbundle/
Disallow: /partnersuche/registration/?user
Disallow: /pf/api/*
Disallow: /pur/
Disallow: /suche/
Disallow: /vergleiche/graphs/
Disallow: /vergleiche/*?xpreview*

History

Changes recorded since June 2026.

No change recorded.

In context

Germany
71.7% block at least one AI crawler
Behind Cloudflare
No
Same group