CrawlerPulse
CrawlerPulse / North America / United States / healthday.com

HealthDay News

healthday.com · United States · measured 9 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 9 Oct 2026, 516 bytes. 4 rule groups name AI crawlers; shown first.

User-agent: GPTBot
Allow: /

User-agent: ClaudeBot
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Google-Extended
Allow: /

User-agent: *
Allow: /api/v1/collections/
Sitemap: https://www.healthday.com/sitemap.xml
Sitemap: https://www.healthday.com/news_sitemap.xml
LLMS: https://www.healthday.com/llms.txt
Disallow: /social-graphics/key-takeaways-and-wtmfy-dont-delete
Disallow: /meta

User-agent: Klaviyo
Allow: /api/v1/collections/
Allow: /

User-agent: Ahref
Disallow: /

User-agent: Dotbot
Disallow: /

History

Changes recorded since June 2026.

No change recorded.

In context

United States
63.2% block at least one AI crawler
Behind Cloudflare
Yes