jiji.com · Japan · Jiji Tsushin(時事通信社) · measured 8 Oct 2026
As read on 8 Oct 2026, 5,411 bytes. 19 rule groups name AI crawlers; shown first.
User-agent: AmazonBot Disallow: / User-agent: anthropic-ai Disallow: / User-agent: AppleBot-Extended Disallow: / User-agent: Bytespider Disallow: / User-agent: CCBot Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Claude-Web Disallow: / User-agent: cohere-ai Disallow: / User-agent: Diffbot Disallow: / User-agent: FacebookBot Disallow: / User-agent: Google-Extended Disallow: / User-agent: GPTBot Disallow: / User-agent: Meta-ExternalAgent Disallow: / User-agent: OAI-SearchBot Disallow: / User-agent: omgili Disallow: / User-agent: omgilibot Disallow: / User-agent: PerplexityBot Disallow: / User-agent: Timpibot Disallow: / User-agent: * Disallow: /cgi-bin/ Disallow: /*.imageparts$ Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ Allow: /hall/ Allow: /service/ Allow: /c_profile/ Allow: /jinji/ Sitemap: https://www.jiji.com/sitemap.xml User-agent: Mediapartners-Google Allow: / Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-agent: Mediapartners Allow: / Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-agent: Googlebot-Image Disallow: /cgi-bin/ Disallow: /jc/m_ Disallow: /*imageparts$ Disallow: /*.gif$ Disallow: /commodi/ Disallow: /forex/ Allow: /*.png$ Allow: /*.jpg$ Allow: / User-Agent: Googlebot Disallow: /cgi-bin/ Disallow: /jc/m_ Disallow: /jc/zc_forward Disallow: /jc/sake Disallow: /jc/search Disallow: /*.gif$ Disallow: /*imageparts$ Disallow: /commodi/ Disallow: /forex/ Allow: / User-Agent: msnbot Crawl-delay: 1 Disallow: /cgi-bin/ Disallow: /jc/m_ Disallow: /jc/zc_forward Disallow: /jc/sake Disallow: /jc/search Disallow: /*.gif$ Disallow: /*.png$ Disallow: /*imageparts$ Disallow: /commodi/ Disallow: /forex/ Allow: / User-agent: bingbot Crawl-delay: 1 Disallow: /cgi-bin/ Disallow: /jc/m_ Disallow: /jc/zc_forward Disallow: /jc/sake Disallow: /jc/search Disallow: /*.gif$ Disallow: /*.png$ Disallow: /*imageparts$ Disallow: /commodi/ Disallow: /forex/ Allow: / User-Agent: Baiduspider+(+http://www.baidu.jp/spider/) Allow: / Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-Agent: BaiduImagespider(+http://www.baidu.jp/spider/) Allow: / Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-agent: Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html) Allow: / Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-agent: Mozilla/5.0 (compatible; Baiduspider/3.0; +http://www.baidu.com/search/spider.html) Allow: / Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-agent: Mozilla/5.0 (compatible; Baiduspider/4.0; +http://www.baidu.com/search/spider.html) Allow: / Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-agent: Mozilla/5.0 (compatible; Baiduspider/5.0; +http://www.baidu.com/search/spider.html) Allow: / Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-agent: Baiduspider/2.0 Allow: / Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-agent: Baiduspider/3.0 Allow: / Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-agent: Baiduspider/4.0 Allow: / Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-agent: Baiduspider/5.0 Allow: / Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-agent: Baiduspider/2.0+(+http://www.baidu.com/search/spider.htm) Allow: / Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-agent: Slurp Allow: / Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-agent: Yahoo! Slurp Allow: / Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-agent: Yeti/1.0 (NHN Corp.; http://help.naver.com/robots/) Allow: / Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-agent: Y!J Allow: / Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-agent: popIn_Agent Allow: / Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-agent: Applebot Allow: / Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-agent: Twitterbot Disallow: /cgi-bin/ Disallow: /jc/m_ Disallow: /jc/zc_forward Disallow: /jc/sake Disallow: /jc/search Disallow: /*.gif$ Disallow: /*imageparts$ Disallow: /commodi/ Disallow: /forex/ Allow: / User-agent: Crowsnest Allow: / Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-agent: logly Allow: / Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-agent: Clipper Allow: / Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-agent: GrapeshotCrawler Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-agent: grapeshot Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-agent: Zucks Recommend Engine Crawler Bot Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-agent: Opebot-v2-beta (http://www.1plusx.com) Allow: / Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-agent: cXensebot Allow: / Disallow: /jc/m_ Disallow: /commodi/ Disallow: /forex/ User-agent: Y!J-BRJ/YATS crawler Disallow: / User-agent: Claude-SearchBot Disallow: / User-agent: Claude-User Disallow: / User-agent: cohere-training-data-crawler Disallow: / User-agent: ICC-Crawler Disallow: / User-agent: magpie-crawler Disallow: / User-agent: Meta-ExternalFetcher Disallow: / User-agent: Perplexity-ai Disallow: / User-agent: Perplexity-User Disallow: / User-agent: SBIntuitionsBot Disallow:/ User-agent: Scrapy Disallow: / User-agent: Webzio Disallow: / User-agent: Webzio-Extended Disallow: /
Changes recorded since June 2026.
No change recorded.