CrawlerPulse
CrawlerPulse / Europe / France / courrierinternational.com

COURRIER INTERNATIONAL

courrierinternational.com · France · Groupe Le Monde · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Blocked
OpenAI
Blocked
Anthropic
Blocked
Perplexity
Blocked
Meta
Blocked
Common Crawl
Blocked
ByteDance
Blocked
Apple
Blocked
Amazon
Blocked

robots.txt

As read on 8 Oct 2026, 6,985 bytes. 2 rule groups name AI crawlers; shown first.

User-agent: 5erue
User-agent: adequat
User-agent: adequat-systems
User-agent: AI2Bot
User-agent: Amazonbot
User-agent: amazon-kendra
User-agent: amazon-QBusiness
User-agent: anthropic-ai
User-agent: Applebot
User-agent: Applebot-Extended
User-agent: asknread.com
User-agent: Bytespider
User-agent: CCBot
User-agent: ChatGPT-User
User-agent: Cision
User-agent: Claude-Web
User-agent: ClaudeBot
User-agent: coexel
User-agent: cohere-training-data-crawler
User-agent: Diffbot
User-agent: DuckAssistBot
User-agent: FacebookBot
User-agent: flipboard
User-agent: Google-Extended
User-agent: GPTBot
User-agent: grub-client
User-agent: infoseek
User-agent: Jetbot
User-agent: k2spider
User-agent: Kangaroo Bot
User-agent: kbcrawl
User-agent: leadbox
User-agent: libwww
User-agent: linkfluence
User-agent: Meltwater
User-agent: mention
User-agent: Meta-ExternalAgent
User-agent: Meta-ExternalFetcher
User-agent: MSIECrawler
User-agent: mytwip
User-agent: Newzbin
User-agent: Offline Explorer
User-Agent: omgili
User-Agent: omgilibot
User-agent: opinion-tracker
User-agent: PanguBot
User-Agent: PerplexityBot
User-agent: proxem
User-agent: Qwam content intelligence
User-agent: scoop.it
User-agent: score3
User-agent: sitecheck.internetseer.com
User-agent: Synthesio
User-agent: Talkwater
User-agent: Teleport
User-agent: TeleportPro
User-agent: Timpibot
User-agent: trendybuzz
User-agent: vecteurplus
User-agent: verticalsearch
User-agent: vsw
User-agent: WebCopier
User-agent: WebStripper
User-agent: Webzio-Extended
User-agent: wget
User-agent: winello
User-agent: YouBot
User-agent: Youmag
User-agent: Zealbot
Disallow: /

User-agent: 5emeRue
User-agent: ACQUIRE MEDIA
User-agent: ACTIV Financial (CME Group)
User-agent: AlphaSense
User-agent: AmiSoftware
User-agent: archive.org_bot
User-agent: Archive-It
User-agent: ArgClrInt
User-agent: Ask n read
User-agent: Augure
User-agent: auramundi
User-agent: AwarioRssBot
User-agent: AwarioSmartBot
User-agent: Barchart.com
User-agent: BattleFin
User-agent: Bernin IT
User-agent: Blackboard Safeassign
User-agent: BLP_bbot
User-agent: bluematrix
User-agent: Brandwatch
User-agent: Briefcase.news
User-agent: Buck
User-agent: Bytespider
User-agent: CCBo
User-agent: CikisiBot
User-agent: Coexel
User-agent: cohere-ai
User-agent: Comtex News Network
User-agent: ConveraCrawler
User-agent: Copyright Licensing Agency
User-agent: Corporama
User-agent: D&B Hoovers
User-agent: Data Expression
User-agent: Data Observer
User-agent: Dataminr
User-agent: Dealogic
User-agent: Diffbot
User-agent: Digimind
User-agent: DirectFN
User-agent: Dun & Bradstreet
User-agent: Dun & Bradstreet - D&B ESG Intelligence
User-agent: Dun & Bradstreet Data Marketplace
User-agent: Eagle Alpha
User-agent: ecoresearch
User-agent: ellisphere
User-agent: FactSet
User-agent: FeedCheck
User-agent: FeedReader
User-agent: Feedspot
User-agent: Fitch Solutions
User-agent: Founder Apabi
User-agent: Freshbot
User-agent: FriendlyCrawler
User-agent: Gnowit
User-agent: GnowitNewsbot
User-agent: Ground News
User-agent: ia_archiver
User-agent: ICE Connect Desktop Solution
User-agent: ICE Data Services
User-agent: IHS Markit
User-agent: ImageSift
User-agent: InMédia Technologies
User-agent: Innguma
User-agent: Inoreader
User-agent: ISI Emerging Markets
User-agent: KB Crawl SAS
User-agent: Knowings
User-agent: Koyfin
User-agent: Launchmetrics
User-agent: LexisNexis
User-agent: Liana
User-agent: magpie-crawler
User-agent: Make/production
User-agent: MarketResearch.com
User-agent: MarketWatch
User-agent: MarketWise
User-agent: Markit Digital
User-agent: Mediatoolkitbot
User-agent: moduleQ
User-agent: MONITIO
User-agent: MoodleBot
User-agent: Moody's
User-agent: Moreover
User-agent: MORNINGSTAR
User-agent: MuckRack
User-agent: Netvibes
User-agent: news-api.org
User-agent: Newslitbot
User-agent: NewsNow
User-agent: Northern Light
User-agent: Opinion-tracker
User-agent: Orbis
User-agent: Opoint
User-agent: Paqlebot
User-agent: Press Monitor Europe
User-agent: PressEngineBot D
User-agent: PriberamBot
User-agent: QuoteMedia
User-agent: QWAM CONTENT INTELLIGENCE
User-agent: RankurBot
User-agent: RavenPack
User-agent: ReportLinker
User-agent: Research & Markets
User-agent: S&P Capital IQ
User-agent: S&P Global Market Intelligence
User-agent: S&P Global Marketplace
User-agent: scoopit
User-agent: scoopit-crawler
User-agent: scpitspi-rs
User-agent: semantic-visions.com
User-agent: semantic-visions.com crawler
User-agent: SemrushBot
User-agent: SentiBot
User-agent: SentiOne
User-agent: Signal Insights
User-agent: Sindup
User-agent: smartkarma
User-agent: Sociallymap
User-agent: spotter
User-agent: Sprinklr
User-agent: squirrobot
User-agent: Statista
User-agent: STOCKBOARD
User-agent: telpress.it
User-agent: Thomson Reuters
User-agent: Thomson Reuters WestLaw
User-agent: TraderPlanet
User-agent: Tradingcharts.com
User-agent: TradingView
User-agent: Transform Your API
User-agent: trendeo
User-agent: trendictionbot
User-agent: TurnitinBot
User-agent: um-FC
User-agent: um-IC
User-agent: um-LN
User-agent: UptimeRobot
User-agent: Vable Ltd
User-agent: VISIBRAIN
User-agent: webzio
User-agent: YaK
User-agent: YCHARTS
User-agent: YouBot
User-agent: Zite
Disallow: /

User-agent: *
Crawl-delay: 10
Disallow: /includes/
Disallow: /misc/
Disallow: /modules/
Disallow: /profiles/
Disallow: /scripts/
Disallow: /themes/
Disallow: /CHANGELOG.txt
Disallow: /cron.php
Disallow: /INSTALL.mysql.txt
Disallow: /INSTALL.pgsql.txt
Disallow: /INSTALL.sqlite.txt
Disallow: /install.php
Disallow: /INSTALL.txt
Disallow: /LICENSE.txt
Disallow: /MAINTAINERS.txt
Disallow: /update.php
Disallow: /UPGRADE.txt
Disallow: /xmlrpc.php
Disallow: /admin/
Disallow: /comment/reply/
Disallow: /filter/tips/
Disallow: /node/add/
Disallow: /ci/async/
Disallow: /search/
Disallow: /users/
Disallow: /user/register
Disallow: /user/password
Disallow: /user/login
Disallow: /user/logout
Disallow: /inscription
Disallow: /login?
Disallow: /sfuser/deconnexion
Disallow: /?score*
Disallow: /jeu/puzzle/score*
Disallow: /article/audio-stream*
Disallow: /?q=admin/
Disallow: /?q=comment/reply/
Disallow: /?q=filter/tips/
Disallow: /?q=node/add/
Disallow: /?q=search/
Disallow: /?q=user/password/
Disallow: /?q=user/register/
Disallow: /?q=user/login/
Disallow: /?q=user/logout/

History

Changes recorded since June 2026.

No change recorded.

In context

France
42.3% block at least one AI crawler
Behind Cloudflare
No
Same group