CrawlerPulse
CrawlerPulse / Asia / Japan / denshobato.com

オホーツクの情報源・伝書鳩WEB

denshobato.com · Japan · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 1,323 bytes. No rule names an AI crawler.

User-agent: *
Disallow: /ssl/
Disallow: /nexus/
Disallow: /dummy/
Disallow: /db_backup/
Disallow: /banner-box/
Disallow: /error/
Disallow: /etc/
Disallow: /filebox/
Disallow: /image/
Disallow: /hosting/
Disallow: /lao/
Disallow: /op/
Disallow: /js/
Disallow: /style/
Disallow: /ipa/
Disallow: /pdf/

User-agent: Aboundexbot
Disallow: /

User-agent: SiteExplorer
Disallow: /

User-agent: WBSearchBot
Disallow: /

User-agent: Mail.Ru
Disallow: /

User-agent: TurnitinBot
Disallow: /

User-agent: Ezooms
Disallow: /

User-agent: MJ12bot
Disallow: /

User-agent: YandexBot
Disallow: /

User-agent: SiteBot
Disallow: /

User-agent: KaloogaBot
Disallow: /

User-agent: findlinks
Disallow: /

User-agent: Exabot
Disallow: /

User-agent: Exabot-Images
Disallow: /

User-agent: AcoonBot
Disallow: /

User-agent: CamontSpider
Disallow: /

User-agent: Yeti
Disallow: /

User-agent: Sogou web spider
Disallow: /

User-agent: SemrushBot
Disallow: /

User-agent: trendiction.de
Disallow: /

User-agent: Wotbox
Disallow: /

User-agent: bing.com
Disallow: /

User-agent: support.paper.li
Disallow: /

User-agent: metauri.com
Disallow: /

User-agent: ahrefs.com
Disallow: /

User-agent: hypefactors.com
Disallow: /

User-agent: baiduspider
Disallow: 3600

User-agent: baiduspider+
Disallow: 3600

User-agent: baiduimagespider
Disallow: 3600

History

Changes recorded since June 2026.

No change recorded.

In context

Japan
32.2% block at least one AI crawler
Behind Cloudflare
No