CrawlerPulse
CrawlerPulse / Asia / Taiwan / storm.mg

Storm News

storm.mg · Taiwan · Storm media Group · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 303 bytes. 1 rule group name AI crawlers; shown first.

User-agent: meta-externalagent
Allow: /

User-agent: *
Allow: /member/premium-plan
Disallow: /exit
Disallow: /s3://*
Disallow: /api/*
Disallow: /preview/
Disallow: /auth/*
Disallow: /member/*
Disallow: /_nuxt/*.js
Sitemap: https://www.storm.mg/sitemaps/1/sitemap.xml
LLMs: https://www.storm.mg/llms.txt

History

Changes recorded since June 2026.

No change recorded.

In context

Taiwan
35.7% block at least one AI crawler
Behind Cloudflare
No