CrawlerPulse
CrawlerPulse / Asia / Japan / asiapress.org

asiapress.org

asiapress.org · Japan · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 558 bytes. No rule names an AI crawler.

<html>
<head><title>404 Not Found</title></head>
<body>
<h1>404 Not Found</h1>
<ul>
<li>Code: NoSuchKey</li>
<li>Message: The specified key does not exist.</li>
<li>Key: robots.txt</li>
<li>RequestId: Z4Q85GTTP68V5GB5</li>
<li>HostId: EOw0r8c3iyu7zUrCw6ikVBaYP10nWUfKd2hOsk5Ti4F58Y7ycbsy+Yx6ogIh012LeDnyrI2a/aszSWHqEGSxEjk2BhXG7XjX</li>
</ul>
<h3>An Error Occurred While Attempting to Retrieve a Custom Error Document</h3>
<ul>
<li>Code: NoSuchKey</li>
<li>Message: The specified key does not exist.</li>
<li>Key: error.html</li>
</ul>
<hr/>
</body>
</html>

History

Changes recorded since June 2026.

No change recorded.

In context

Japan
32.2% block at least one AI crawler
Behind Cloudflare
No