CrawlerPulse
CrawlerPulse / North America / Canada / cp24.com

CP24

cp24.com · Canada · Bell Media · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Blocked
Anthropic
Allowed
Perplexity
Blocked
Meta
Allowed
Common Crawl
Blocked
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 1,226 bytes. 8 rule groups name AI crawlers; shown first.

User-agent: GPTBot
Disallow: /

User-agent: ChatGPT-User
Disallow: /

User-agent: OAI-SearchBot
Disallow: /

User-Agent: PerplexityBot
Disallow: /

User-agent: CCBot
Disallow: /

User-agent: Anthropic-ai
Disallow: /

User-agent: DeepSeekBot
Disallow: /

User-agent: Claude-Web
Disallow: /

Sitemap: https://www.cp24.com/arc/outboundfeeds/sitemap-index/
Sitemap: https://www.cp24.com/arc/outboundfeeds/sitemap-news-index/
Sitemap: https://www.cp24.com/arc/outboundfeeds/sitemap-section-index/
Sitemap: https://www.cp24.com/arc/outboundfeeds/sitemap-video-index/

User-agent: *
Allow: /
Disallow: /search-results/
Disallow: /poll/
Disallow: /logger/
Disallow: /cmlink/
Disallow: /cm/
Disallow: /error/
Disallow: /errorpages/
Disallow: /content/
Disallow: /captcha/
Disallow: /membership/
Disallow: /blogimageupload/
Disallow: /polopolydevelopment/
Disallow: /status/
Disallow: /redirect/
Disallow: /crossdomain.xml
Disallow: /flagcomment/
Disallow: /mobile/
Disallow: /comments-7.*
Disallow: /*?ghd_jid=*
Disallow: /*?cache=*
Disallow: /*?fbclid=*
Disallow: /*-1.*

User-agent: Googlebot-Image
Allow: /resizer/v2/
Disallow: /polopoly_fs/

User-agent: Twitterbot
Allow: /
Disallow: /polopoly_fs/

User-Agent: DeepSeek
Disallow: /

History

Changes recorded since June 2026.

No change recorded.

In context