CrawlerPulse
CrawlerPulse / Asia / Taiwan / newtalk.tw

Newtalk News

newtalk.tw · Taiwan · measured 9 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 9 Oct 2026, 365 bytes. No rule names an AI crawler.

User-agent: *
Content-Signal: search=yes, ai-input=no, ai-train=no
Sitemap: https://newtalk.tw/sitemap.xml
Disallow: /member
Disallow: /member/*
Disallow: /trytest
Disallow: /trytest/*
Disallow: /news/view_content/*
Disallow: /cdn-cgi/*
Disallow: /search
Disallow: /forum/search
Disallow: /forum/search/*
Disallow: /go
Disallow: /download
Disallow: /forum/amp_post

History

Changes recorded since June 2026.

No change recorded.

In context

Taiwan
35.7% block at least one AI crawler
Behind Cloudflare
Yes