CrawlerPulse
CrawlerPulse / North America / United States / fox13news.com

KSTU

fox13news.com · United States · E.W. Scripps Company · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Blocked
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 810 bytes. 1 rule group name AI crawlers; shown first.

User-agent: GPTBot
Disallow: /

User-agent: AmazonAdBot
Allow: /

User-agent: *
Disallow: /search
Disallow: /maintenance/
Disallow: /_wzln
Disallow: /_wzln/
Disallow: /*mailto:*
Disallow: /*@*
Disallow: /*%40*
Disallow: /*email=*
Disallow: /*contact=*
Disallow: /news/mailto:
Disallow: /news/@
Sitemap: https://www.fox13news.com/sitemap.xml
Sitemap: https://www.fox13news.com/sitemap.xml?type=fastchanging
Sitemap: https://www.fox13news.com/sitemap.xml?type=articles
Sitemap: https://www.fox13news.com/sitemap.xml?type=videos

User-agent: ia_archiver
Disallow: /

User-agent: archive.org_bot
Disallow: /

User-agent: wayback
Disallow: /

User-agent: Googlebot
Disallow: /rss/
Disallow: /apple-app-site-association
Disallow: /social-media/link-icon.svg
Disallow: /social-media/yt-icon.svg
Disallow: /taboola/

History

Changes recorded since June 2026.

No change recorded.

In context

United States
63.2% block at least one AI crawler
Behind Cloudflare
No
Same group