CrawlerPulse
CrawlerPulse / Europe / France / slate.fr

slate.fr

slate.fr · France · bo.slate.fr · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Blocked
OpenAI
Blocked
Anthropic
Blocked
Perplexity
Blocked
Meta
Blocked
Common Crawl
Blocked
ByteDance
Blocked
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 4,112 bytes. 18 rule groups name AI crawlers; shown first.

User-agent: anthropic-ai
Disallow: /

User-agent: Bytespider
Disallow: /

User-agent: CCBot
Disallow: /

User-agent: ChatGPT-User
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: Claude-Web
Disallow: /

User-agent: cohere-ai
Disallow: /

User-agent: Diffbot
Disallow: /

User-agent: FacebookBot
Disallow: /

User-agent: Google-Extended
Disallow: /

User-agent: GPTBot
Disallow: /

User-agent: ImagesiftBot
Disallow: /

User-agent: Meta-ExternalAgent
User-agent: meta-externalagent
Disallow: /

User-agent: OAI-SearchBot
Disallow: /

User-Agent: omgili
Disallow: /

User-Agent: omgilibot
Disallow: /

User-Agent: PerplexityBot
Disallow: /

User-agent: YouBot
Disallow: /

User-agent: *
Disallow: /assets/
Disallow: /board/
Disallow: /channels/
Disallow: /components/
Disallow: /controllers/
Disallow: /formatters/
Disallow: /helpers/
Disallow: /javascript/
Disallow: /jobs/
Disallow: /mailers/
Disallow: /models/
Disallow: /policies/
Disallow: /recherche
Disallow: /services/
Disallow: /uploads/
Disallow: /uploaders/
Disallow: /views/
Disallow: /workers/
Allow: /assets/*.css$
Allow: /assets/*.css?
Allow: /assets/*.js$
Allow: /assets/*.js?
Allow: /assets/*.gif
Allow: /assets/*.jpg
Allow: /assets/*.jpeg
Allow: /assets/*.png
Allow: /assets/*.woff$
Allow: /assets/*.woff2$
Allow: /assets/*.ttf$
Allow: /assets/*.eot$
Allow: /assets/*.svg$
Allow: /assets/content/
Allow: /assets/video/
Allow: /assets/mp4/
Allow: /uploads/store/
Disallow: /api/
Disallow: /audio/board/
Disallow: /blognews/
Disallow: /BLOGS_CHRONIQUES/
Disallow: /cdn-cgi/
Disallow: /embed/
Disallow: /plus/
Disallow: /reader/
Disallow: /slate_amazon/
Disallow: /slate_podcasts/
Disallow: /slate_theme_serie/
Disallow: /slate_views_topics/
Disallow: /widgets/
Disallow: /21685585667/
Disallow: /current_user_status
Sitemap: https://www.slate.fr/sitemap.xml

User-agent: 5emeRue
Disallow: /

User-agent: 5erue
Disallow: /

User-agent: adequat
Disallow: /

User-agent: adequat-systems
Disallow: /

User-agent: AmiSoftware
Disallow: /

User-agent: asknread.com
Disallow: /

User-agent: Augure
Disallow: /

User-agent: auramundi
Disallow: /

User-agent: AwarioRssBot
User-agent: AwarioSmartBot
Disallow: /

User-agent: Cision
Disallow: /

User-agent: coexel
Disallow: /

User-agent: ConveraCrawler
Disallow: /

User-agent: Corporama
Disallow: /

User-agent: DataForSeoBot
Disallow: /

User-agent: Digimind
Disallow: /

User-agent: ellisphere
Disallow: /

User-agent: fast
Disallow: /

User-agent: FriendlyCrawler
Disallow: /

User-agent: grub-client
Disallow: /

User-agent: infoseek
Disallow: /

User-agent: Jetbot
Disallow: /

User-agent: k2spider
Disallow: /

User-agent: kbcrawl
Disallow: /

User-agent: Knowings
Disallow: /

User-agent: leadbox
Disallow: /

User-agent: libwww
Disallow: /

User-agent: linkfluence
Disallow: /

User-agent: magpie-crawler
Disallow: /

User-agent: Meltwater
Disallow: /

User-agent: Moreover
Disallow: /

User-agent: MSIECrawler
Disallow: /

User-agent: mytwip
Disallow: /

User-agent: Newzbin
Disallow: /

User-agent: NewsNow
Disallow: /

User-agent: news-please
Disallow: /

User-agent: Offline Explorer
Disallow: /

User-agent: opinion-tracker
Disallow: /

User-agent: peer39_crawler
User-agent: peer39_crawler/1.0
Disallow: /

User-agent: proxem
Disallow: /

User-agent: Qwam content intelligence
Disallow: /

User-agent: scoop.it
Disallow: /

User-agent: score3
Disallow: /

User-agent: Scrapy
Disallow: /

User-agent: Sindup
Disallow: /

User-agent: sitecheck.internetseer.com
Disallow: /

User-agent: Synthesio
Disallow: /

User-agent: Talkwater
Disallow: /

User-agent: Teleport
Disallow: /

User-agent: TeleportPro
Disallow: /

User-agent: trendeo
Disallow: /

User-agent: trendybuzz
Disallow: /

User-agent: TurnitinBot
Disallow: /

User-agent: vecteurplus
Disallow: /

User-agent: verticalsearch
Disallow: /

User-agent: vsw
Disallow: /

User-agent: WebCopier
Disallow: /

User-agent: WebReaper
Disallow: /

User-agent: WebStripper
Disallow: /

User-agent: wget
Disallow: /

User-agent: winello
Disallow: /

User-agent: Zealbot
Disallow: /

History

Changes recorded since June 2026.

No change recorded.

In context

France
42.3% block at least one AI crawler
Behind Cloudflare
Yes