CrawlerPulse
CrawlerPulse / Asia / Japan / ehime-np.co.jp

Ehime Shimbun Online

ehime-np.co.jp · Japan · Ehime Shimbun(愛媛新聞社) · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Blocked
OpenAI
Blocked
Anthropic
Blocked
Perplexity
Blocked
Meta
Blocked
Common Crawl
Blocked
ByteDance
Blocked
Apple
Blocked
Amazon
Blocked

robots.txt

As read on 8 Oct 2026, 4,245 bytes. 19 rule groups name AI crawlers; shown first.

User-agent: GPTBot
Disallow: /

User-agent: ChatGPT-User
Disallow: /

User-agent: Google-Extended
Disallow: /

User-agent: anthropic-ai
Disallow: /

User-agent: CCBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: Claude-Web
Disallow: /

User-agent: PerplexityBot
Disallow: /

User-agent: Bytespider
Disallow: /

User-agent: Diffbot
Disallow: /

User-agent: OAI-SearchBot
Disallow: /

User-agent: FacebookBot
Disallow: /

User-agent: meta-externalagent
Disallow: /

User-agent: cohere-ai
Disallow: /

User-Agent: omgili
Disallow: /

User-Agent: omgilibot
Disallow: /

User-agent: Timpibot
Disallow: /

User-agent: Applebot-Extended
Disallow: /

User-agent: AmazonBot
Disallow: /

User-agent:*
Disallow: /ad/ad_guide/*
Disallow: /online/education/*
Disallow: /online/education_junior/*
Disallow: /online/hiroba/esta/audio/*
Disallow: /member/
Disallow: /houdou/young/
Disallow: /i/
Disallow: /ad/dokusho/*
Disallow: /online/business/market/*
Disallow: /online/sports/paris_olympic_paralympic/*
Disallow: /news/index.html
Disallow: /gyoumu/index.html
Disallow: /tokushuex/*
Disallow: /gougai/index.html
Disallow: /%
Disallow: /cgi-bin/
Disallow: /gyoumu/np-npsub.html
Disallow: /gyoumu/np-npsubex.html
Disallow: /gyoumu/kyozai.html
Disallow: /hptensai/index.html
Disallow: /kijidb/card.html
Disallow: /kijidb/siryou.html
Disallow: /toko/index.html
Disallow: /douga/
Disallow: /tokushuex/index.html
Disallow: /gyoumu/index.html
Disallow: /gougai/index.html
Disallow: /newsmart/smartphone4ios7.html
Disallow: /media/images/news/20180712
Disallow: /AccreteClub_web/member/
Disallow: *menu=olympic2016_list$
Disallow: /online/form/
Allow: /online/form/contact
Disallow: /AccreteClub_web00/
Disallow: /online/enjoy/summer_treasure/download/
Disallow: /wire/google/images/
Disallow: /online/information/contest_toko/2021.html
Disallow: /online/information/contest_toko/complete.html
Disallow: /online/information/contest_toko/work/
Disallow: /online/information/contest_toko/work2019/
Disallow: /online/information/contest_toko/work2020/
Disallow: /online/information/contest_toko/work2021/
Disallow: /online/information/contest_toko/work_meiji/
Disallow: /rensai/
Disallow: /special/egao_sports/kyogi/kokutai_para/
Disallow: /pr/db/
Disallow: /db/
Disallow: /culture/fuji/matsuyama/2021/03/316.html
Disallow: /culture/fuji/matsuyama/2018/03/post-105.html
Disallow: /culture/fuji/matsuyama/2017/02/post-105.html
Disallow: /culture/fuji/matsuyama/2017/03/post-105.html
Disallow: /culture/fuji/experience/2017/03/post-39.html
Disallow: /online/information/admission_gift/2022_index.html
Disallow: /article/news202101120088
Disallow: /article/news202111160032
Disallow: /online/information/ad/file/enpad_line.pdf
Disallow: /online/wedding/
Disallow: /online/news/amp_stories/kotobanotikara/*
Disallow: /online/election/tokushu/2022sanin.html
Disallow: /article/news201810110096
Disallow: /article/news201710239191
Disallow: /online/riq/*
Disallow: /online/enpsc/*
Disallow: /e4/*
Disallow: /online/jibunshi/*
Disallow: /special/egao_sports/beijing_olympic_paralympic/*
Disallow: /special/egao_sports/tokyo_olympic_paralympic/*
Disallow: /online/sports/paris_olympic_paralympic/*
Disallow: /online/data/*
Disallow: /online/life/cancer/event/index.html
Disallow: /online/webtools/*

User-agent: bingbot
Crawl-delay: 180

User-agent: BingPreview
Disallow: /

User-agent: BLP_bbot
Disallow: /
Crawl-delay: 180

User-agent: BLP_bbot/0.1
Disallow: /
Crawl-delay: 180

User-agent: Claude-SearchBot
Disallow: /

User-agent: Claude-User
Disallow: /

User-agent: Perplexity-ai
Disallow: /

User-agent: Perplexity‑User
Disallow: /

User-agent: MJ12bot
Disallow: /

User-agent: PiplBot
Disallow: /

User-agent: meta-externalfetcher
Disallow: /

User-agent: cohere-training-data-crawler
Disallow: /

User-agent: ICC-Crawler
Disallow: /

User-agent: Webzio
Disallow: /

User-agent: Webzio-extended
Disallow: /

User-agent: magpie-crawler
Disallow: /

User-agent: Scrapy
Disallow: /

User-agent: Applebot
Crawl-delay: 180
Sitemap: https://www.ehime-np.co.jp/sitemap.xml

History

Changes recorded since June 2026.

No change recorded.

In context

Japan
32.2% block at least one AI crawler
Behind Cloudflare
No