CrawlerPulse
CrawlerPulse / Europe / France / ouest-france.fr

Ouest France

ouest-france.fr · France · Groupe SIPA - Ouest-France · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Blocked
OpenAI
Blocked
Anthropic
Blocked
Perplexity
Blocked
Meta
Blocked
Common Crawl
Blocked
ByteDance
Blocked
Apple
Blocked
Amazon
Blocked

robots.txt

As read on 8 Oct 2026, 9,822 bytes. 25 rule groups name AI crawlers; shown first.

User-agent: ChatGPT-User
User-agent: Claude-User
User-agent: Claude-SearchBot
User-agent: MistralAI-User
User-agent: PerplexityBot
User-agent: Perplexity-User
User-agent: DuckAssistBot
User-agent: Gemini-DeepResearch
User-agent: Applebot-Extended
User-agent: Google-Extended
User-agent: Google-NotebookLM
User-agent: FacebookBot
User-agent: Amazonbot
User-agent: Bytespider
User-agent: OAI-SearchBot
User-agent: Meta-ExternalFetcher
User-agent: Meta-WebIndexer
User-agent: MistralAI-Index
Disallow: /shopping/code-promo/
Allow: /shopping/
Allow: /tourisme/
Allow: /qui-sommes-nous/
Allow: /publi-info/
Allow: /auvergne-rhone-alpes/
Allow: /bourgogne-franche-comte/
Allow: /centre-val-de-loire/
Allow: /corse/
Allow: /grand-est/
Allow: /hauts-de-france/
Allow: /ile-de-france/
Allow: /mayotte/
Allow: /nouvelle-aquitaine/
Allow: /nouvelle-caledonie/
Allow: /polynesie/
Allow: /provence-alpes-cote-dazur/
Allow: /region-guadeloupe/
Allow: /region-guyane/
Allow: /region-la-reunion/
Allow: /region-martinique/
Allow: /region-occitanie/
Allow: /saint-barthelemy/
Allow: /saint-pierre-et-miquelon/
Allow: /tahiti/
Allow: /wallis-et-futuna/
Allow: /*/$
Disallow: /*?

User-agent: Amazonbot
Disallow: /

User-agent: AmazonBot
Disallow: /

User-agent: anthropic-ai
Disallow: /

User-agent: Applebot-Extended
Disallow: /

User-agent: Bytespider
Disallow: /

User-agent: CCBot
Disallow: /

User-agent: ChatGPT-User
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: Claude-Web
Disallow: /

User-agent: Cohere-ai
Disallow: /

User-agent: DeepSeekBot
Disallow: /

User-agent: Diffbot
Disallow: /

User-agent: Google-Extended
Disallow: /

User-agent: GPTBot
Disallow: /

User-agent: ImageSiftBot
Disallow: /

User-agent: Meta-ExternalAgent
Disallow: /

User-agent: MistralAI-User
Disallow: /

User-agent: OAI-SearchBot
Disallow: /

User-agent: Omgili
Disallow: /

User-agent: PerplexityBot
Disallow: /

User-agent: PetalBot
Disallow: /

User-agent: Timpibot
Disallow: /

User-agent: Timpibot
Disallow: /

User-agent: YouBot
Disallow: /

User-agent: Mediapartners-Google
User-agent: Googlebot
User-agent: Googlebot-Image
User-agent: Googlebot-News
User-agent: Googlebot-Video
User-agent: GoogleOther
User-agent: Adsbot-Google
User-Agent: Storebot-Google
User-Agent: AdsBot-Google-Mobile
User-Agent: APIs-Google
User-agent: Twitterbot
User-agent: Applebot
User-agent: Ouestfrancebot
User-agent: Taboolabot
User-agent: Proximic
User-agent: upday
User-agent: Bingbot
User-agent: exabot
User-agent: Slurp
User-agent: ia_archiver
User-agent: grapeshot
User-agent: facebookexternalhit
User-agent: Qwantbot
User-agent: cXensebot
User-agent: AmazonAdBot
User-agent: IbouBot
User-agent: bnf.fr_bot
Sitemap: https://www.ouest-france.fr/sitemap.xml
Sitemap: https://www.ouest-france.fr/googlenews.xml
Disallow: */mail?id=*
Disallow: */newsletters/bundles/*
Disallow: */test-de-publication-*
Disallow: /*%2525
Disallow: /*&edition=*
Disallow: /*&fbclid=*
Disallow: /*&mediego*
Disallow: /*&parution=*
Disallow: /*?datasport*
Disallow: /*?dateCompetition=*
Disallow: /*?dialog=*
Disallow: /*?edition=*
Disallow: /*?fbclid=*
Disallow: /*?login
Disallow: /login?redirect=*
Disallow: /logout?redirect=*
Disallow: /*?logout
Disallow: /*?mediego*
Disallow: /*?parution=*
Disallow: /admin/
Disallow: /ajax_comment/
Disallow: /balades/
Disallow: /balades?
Disallow: /cinema/film/
Disallow: /comment/reply/
Disallow: /communaute/galerie-photo/*/ajout
Disallow: /economie/entreprises/centrale-des-marches
Disallow: /economie/entreprises/evenements-des-entreprises
Disallow: /economie/entreprises/offres-emploi
Disallow: /espace-contributions/
Disallow: /espace-debat/utilisateur/
Disallow: /espace-debat/commentaires/
Disallow: /etudiant/job-formation/
Disallow: /examens/
Disallow: /filter/tips/
Disallow: /install.php
Disallow: /jeux-olympiques/resultats/*/?*
Disallow: /mail?type*
Disallow: /meteo/*/?date=*
Disallow: /obseques/search/
Disallow: /publication-statics/
Disallow: /recherche/
Disallow: /rpc/
Disallow: /scribblelive
Disallow: /search/
Disallow: /share/
Disallow: /shopping/code-promo/redirection
Disallow: /sport/running/*/calendrier-des-courses/?*
Disallow: /stories/
Disallow: /update.php
Disallow: /user
Disallow: /user/
Disallow: /user/login/
Disallow: /user/logout/
Disallow: /user/password/
Disallow: /user/register?*
Disallow: /xmlrpc.php

User-agent: Googlebot-News
Disallow: /cinema/salle/
Disallow: /espace-debat/
Disallow: /communaute/
Disallow: /shopping/code-promo/

User-agent: exabot
User-agent: Slurp
User-agent: ia_archiver
Crawl-Delay: 20

User-agent: 5emeRue
Disallow: /

User-agent: 5erue
Disallow: /

User-agent: adequat
Disallow: /

User-agent: adequat-systems
Disallow: /

User-agent: AgentCore-Browser
Disallow: /

User-agent: Ai2Bot-Dolma
Disallow: /

User-agent: AI2Bot/v2.0
Disallow: /

User-agent: Aihitdata.com
Disallow: /

User-agent: Amzn-SearchBot
Disallow: /

User-agent: Amzn-User
Disallow: /

User-agent: AmiSoftware
Disallow: /

User-agent: amazon-kendra
Disallow: /

User-agent: Argus
Disallow: /

User-agent: Asknread
Disallow: /

User-agent: asknread.com
Disallow: /

User-agent: Augure
Disallow: /

User-agent: auramundi
Disallow: /

User-agent: bedrockbot/3.0
Disallow: /

User-agent: Bloodhound
Disallow: /

User-agent: ChatGPT-Agent
Disallow: /

User-agent: ChatGLM-Spider
Disallow: /

User-agent: Channel3Bot
Disallow: /

User-agent: Cision
Disallow: /

User-agent: Claude-SearchBot
Disallow: /

User-agent: Claude-User
Disallow: /

User-agent: coexel
Disallow: /

User-agent: ConveraCrawler
Disallow: /

User-agent: Corporama
Disallow: /

User-agent: CotoyogiBot
Disallow: /

User-agent: cydralspider
Disallow: /

User-agent: DemoSFBot
Disallow: /

User-agent: Digimind
Disallow: /

User-agent: DownloadNinja
Disallow: /

User-agent: downloadexpress
Disallow: /

User-agent: DuckAssistBot
Disallow: /

User-agent: EDD
Disallow: /

User-agent: ellisphere
Disallow: /

User-agent: eureka
Disallow: /

User-agent: Europresse
Disallow: /

User-agent: Explore
Disallow: /

User-agent: Factiva
Disallow: /

User-agent: Fasterfox
Disallow: /

User-agent: Fetch
Disallow: /

User-agent: gammaSpider
Disallow: /

User-agent: Gemini-DeepResearch
Disallow: /

User-agent: Google-NotebookLM
Disallow: /

User-agent: GoogleAgent-Shopping
Disallow: /

User-agent: Google-CloudVertextBot
Disallow: /

User-agent: grub-client
Disallow: /

User-agent: Grok Bot
Disallow: /

User-agent: HTTrack
Disallow: /

User-agent: ia_archiver
Disallow: /

User-agent: IAskBot
Disallow: /

User-agent: ia_archiver-web.archive.org
Disallow: /

User-agent: ICC-Crawler
Disallow: /

User-agent: img2dataset
Disallow: /

User-agent: indexer
Disallow: /

User-agent: infoseek
Disallow: /

User-agent: Jetbot
Disallow: /

User-agent: JuspayVisaInnovationCentreAgentApp
Disallow: /

User-agent: k2spider
Disallow: /

User-agent: Kantar
Disallow: /

User-agent: kbcrawl
Disallow: /

User-agent: KernelSearchBot
Disallow: /

User-agent: Knowings
Disallow: /

User-agent: KunatoBot
Disallow: /

User-agent: larbin
Disallow: /

User-agent: leadbox
Disallow: /

User-agent: libwww
Disallow: /

User-agent: LightSiteBot
Disallow: /

User-agent: LinerBot
Disallow: /

User-agent: linkfluence
Disallow: /

User-agent: linko
Disallow: /

User-agent: manageo
Disallow: /

User-agent: Manus.im
Disallow: /

User-agent: mediacompil
Disallow: /

User-agent: Meltwater
Disallow: /

User-agent: mention
Disallow: /

User-agent: Meta-ExternalFetcher
Disallow: /

User-agent: Meta-WebIndexer
Disallow: /

User-agent: Mistral.ai/crawler
Disallow: /

User-agent: MistralAI-Index
Disallow: /

User-agent: MistralAI-training
Disallow: /

User-agent: Moreover
Disallow: /

User-agent: Muse Agent
Disallow: /

User-agent: MSIECrawler
Disallow: /

User-agent: mytwip
Disallow: /

User-agent: newscan-online
Disallow: /

User-agent: NewsNow
Disallow: /

User-agent: Newzbin
Disallow: /

User-agent: NPBot
Disallow: /

User-agent: ObjectsSearch
Disallow: /

User-agent: Offline Explorer
Disallow: /

User-agent: opinion-tracker
Disallow: /

User-agent: Ora.ai​
Disallow: /

User-agent: PanguBot
Disallow: /

User-agent: Perplexity-User
Disallow: /

User-agent: Pimptrain
Disallow: /

User-agent: proxem
Disallow: /

User-agent: ProRataIncBot
Disallow: /

User-agent: QualifiedBot
Disallow: /

User-agent: QuepasaCreep
Disallow: /

User-agent: Qwam-content-intelligence
Disallow: /

User-agent: Raven
Disallow: /

User-agent: readability.com
Disallow: /

User-agent: RyeBot
Disallow: /

User-agent: scoop.it
Disallow: /

User-agent: score3
Disallow: /

User-agent: SeekrBot
Disallow: /

User-agent: Sindup
Disallow: /

User-agent: sitecheck.internetseer.com
Disallow: /

User-agent: SiteSnagger
Disallow: /

User-agent: spotter
Disallow: /

User-agent: Synthesio
Disallow: /

User-agent: Talkwater
Disallow: /

User-agent: Teleport
Disallow: /

User-agent: TikTokSpider
Disallow: /

User-agent: TikTokSpider
Disallow: /

User-agent: TeleportPro
Disallow: /

User-agent: trendeo
Disallow: /

User-agent: trendybuzz
Disallow: /

User-agent: TunitinBot
Disallow: /

User-agent: up2news
Disallow: /

User-agent: vecteurplus
Disallow: /

User-agent: VelenCrawler
Disallow: /

User-agent: Verif
Disallow: /

User-agent: verticalsearch
Disallow: /

User-agent: VisaLaneAgent
Disallow: /

User-agent: VisaLoopAgent
Disallow: /

User-agent: vsw
Disallow: /

User-agent: wapspider
Disallow: /

User-agent: WARDBot
Disallow: /

User-agent: WebCopier
Disallow: /

User-agent: WebReaper
Disallow: /

User-agent: WebStripper
Disallow: /

User-agent: WebZinger
Disallow: /

User-agent: WebZIP
Disallow: /

User-agent: Wget
Disallow: /

User-agent: winello
Disallow: /

User-agent: WPBot
Disallow: /

User-agent: WrtnBot
Disallow: /

User-agent: Youmag
Disallow: /

User-agent: ZanistaBot
Disallow: /

User-agent: Zealbot
Disallow: /

User-agent: Zite
Disallow: /

User-agent: ZyBORG
Disallow: /

User-agent: *
Disallow: /

History

Changes recorded since June 2026.

No change recorded.

In context

France
42.3% block at least one AI crawler
Behind Cloudflare
No
Same group
gazettevaldoise.fr, courrierdelouest.fr, lepaysdauge.fr, journallamarne.fr, lecourrierdesyvelines.fr, lemainelibre.fr, lepaysbriard.fr, presseocean.fr