CrawlerPulse
CrawlerPulse / North America / United States / columbusalive.com

Columbus Alive

columbusalive.com · United States · USAToday · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 2,394 bytes. No rule names an AI crawler.

User-agent: Googlebot-News
Disallow: /story/sponsor-story/
Disallow: /picture-gallery/sponsor-story/
Disallow: /videos/sponsor-story/
Disallow: /longform/sponsor-story/
Disallow: /pages/interactives/sponsor-story/
Disallow: /interactives/sponsor-story/
Disallow: /videos/embed/

User-agent: *
Disallow: /errors
Disallow: /interactive/
Disallow: /userauth/
Disallow: /ugc/
Disallow: /feeds/
Disallow: /services/
Disallow: /facebook/
Disallow: /version-info/
Disallow: /longform/draft/
Disallow: /story/draft/
Disallow: /topic/*/smart/
Disallow: /search
Disallow: /module-showcase/
Disallow: /newsletter/
Disallow: /blended-newsletter/
Disallow: /story/nletter/
Disallow: /sports/services/photos/
Disallow: /optimus
Disallow: /ux-train
Disallow: /story/advisory/
Disallow: /*?refresh
Disallow: /*template=printart
Disallow: /.cache/
Disallow: /a/
Disallow: /advertising/
Disallow: /apps/pbcs.dll/art_tips
Disallow: /apps/pbcs.dll/classifieds
Disallow: /apps/pbcs.dll/css/
Disallow: /apps/pbcs.dll/error
Disallow: /apps/pbcs.dll/events
Disallow: /apps/pbcs.dll/exec
Disallow: /apps/pbcs.dll/index
Disallow: /apps/pbcs.dll/js/
Disallow: /apps/pbcs.dll/misc
Disallow: /apps/pbcs.dll/netguest
Disallow: /apps/pbcs.dll/ptshowguide
Disallow: /apps/pbcs.dll/ptshowguideitem
Disallow: /apps/pbcs.dll/related
Disallow: /apps/pbcs.dll/search
Disallow: /apps/pbcs.dll/temaoversikt
Disallow: /apps/pbcsad.dll
Disallow: /apps/pbcsi.dll
Disallow: /apps/rub.dll
Disallow: /binary/
Disallow: /blotter/
Disallow: /carousel/
Disallow: /default_rewrites.ini
Disallow: /doubleclick/
Disallow: /error
Disallow: /external-search/
Disallow: /factbox/
Disallow: /images/
Disallow: /includes/
Disallow: /interact/
Disallow: /interstitial/
Disallow: /localdata.ini
Disallow: /logs/
Disallow: /mal/
Disallow: /navigation/
Disallow: /notices/
Disallow: /overlay/
Disallow: /pagination/
Disallow: /peel/
Disallow: /publicus.ini
Disallow: /relatedimage/
Disallow: /scr/
Disallow: /search/
Disallow: /section?template=emailfriend
Disallow: /shareable/
Disallow: /skyslider/
Disallow: /submit/
Disallow: /templates/
Disallow: /tmp/
Disallow: /verticals/
Disallow: /w/
Disallow: /xsendmail.ini
Sitemap: https://www.columbusalive.com/news-sitemap.xml
Sitemap: https://www.columbusalive.com/web-sitemap-index.xml
Sitemap: https://www.columbusalive.com/video-sitemap-index.xml

History

Changes recorded since June 2026.

2026-10-08Blocked → AllowedGoogle-Extended

In context

United States
63.2% block at least one AI crawler
Behind Cloudflare
No
Same group