CrawlerPulse
CrawlerPulse / North America / United States / bleacherreport.com

Bleacher Report

bleacherreport.com · United States · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Blocked
OpenAI
Blocked
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Blocked
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 1,175 bytes. 5 rule groups name AI crawlers; shown first.

User-Agent: GPTBot
Disallow: /

User-Agent: Google-Extended
Disallow: /

User-Agent: CCBot
Disallow: /

User-Agent: FacebookBot
Disallow: /

User-Agent: anthropic-ai
Disallow: /

Sitemap: https://bleacherreport.com/sitemaps
Sitemap: https://bleacherreport.com/sitemaps/categories/scores
Sitemap: https://bleacherreport.com/sitemaps/google-news
Sitemap: https://bleacherreport.com/sitemaps/writers
Sitemap: https://bleacherreport.com/sitemaps/creators

User-Agent: *
Disallow: /AAMSZ=160x600
Disallow: /AAMSZ=300x250
Disallow: /AAMSZ=728x90
Disallow: /admin
Disallow: /account
Disallow: /accounts
Disallow: /activities
Disallow: /add_to_blacklist
Disallow: /archives
Disallow: /api
Disallow: /bets/iframe
Disallow: /login
Disallow: /long_poll
Disallow: /new_writer
Disallow: /newsletter_subscriptions
Disallow: /signup
Disallow: /users
Disallow: /users/undefined
Disallow: /users/ranking
Disallow: /*/archives
Disallow: /oembed
Disallow: /user_post
Disallow: /embedded
Disallow: /mobile
Disallow: /_not-found
Disallow: /creator-login
Disallow: /up/elb
Disallow: /static-pages
Disallow: */scores?date=*
Disallow: /session-context/headwaiter/
Disallow: /videos
Disallow: /share

History

Changes recorded since June 2026.

No change recorded.

In context

United States
63.2% block at least one AI crawler
Behind Cloudflare
No