lesechos.fr · France · measured 8 Oct 2026
As read on 8 Oct 2026, 15,868 bytes. 21 rule groups name AI crawlers; shown first.
User-agent: Amazonbot Disallow: / User-agent: AmazonBot Disallow: / User-agent: anthropic-ai Disallow: / User-agent: Applebot-Extended Disallow: / User-agent: Bytespider Disallow: / User-agent: CCBot Disallow: / User-agent: ChatGPT-User Disallow: / Allow: /thema User-agent: Claude-web Disallow: / User-agent: ClaudeBot Disallow: / User-agent: cohere-ai Disallow: / User-agent: DeepSeekBot Disallow: / User-agent: FacebookBot Disallow: / User-agent: Google-Extended Disallow: / Allow: /thema User-agent: GPTBot Disallow: / Allow: /thema User-agent: ImageSiftBot Disallow: / User-agent: meta-externalagent Disallow: / User-agent: MistralAI-User Disallow: / User-agent: OAI-SearchBot Disallow: / Allow: /thema User-agent: omgili Disallow: / User-agent: omgilibot Disallow: / User-agent: PerplexityBot Disallow: / Allow: /thema User-agent: * Disallow: /internal Disallow: /recherche User-agent: 5emeRue Disallow: / User-agent: 5erue Disallow: / User-agent: adequat Disallow: / User-agent: AI2Bot/v2.0 Disallow: / User-agent: Aihitdata.com Disallow: / User-agent: Alexibot Disallow: / User-agent: AlphaSense Disallow: / User-agent: AlvinetSpider Disallow: / User-agent: amazon-kendra Disallow: / User-agent: AmiSoftware Disallow: / User-agent: Anomura Disallow: / User-agent: Antenne Hatena Disallow: / User-agent: ApifyBot Disallow: / User-agent: ApifyWebsiteContentCrawler Disallow: / User-agent: ApocalXExplorerBot Disallow: / User-agent: Applebot Disallow: / User-agent: Aranet-SearchBot Disallow: / User-agent: archive.org_bot Disallow: / User-agent: Argus Disallow: / User-agent: Ask n read Disallow: / User-agent: asknread.com Disallow: / User-agent: asterias Disallow: / User-agent: atlassian-bot Disallow: / User-agent: Augure Disallow: / User-agent: auramundi Disallow: / User-agent: AwarioRssBot Disallow: / User-agent: AwarioSmartBot Disallow: / User-agent: AzureAI-SearchBot Disallow: / User-agent: BackDoorBot/1.0 Disallow: / User-agent: Barchart.com Disallow: / User-agent: BattleFin Disallow: / User-agent: bedrockbot/3.0 Disallow: / User-agent: Bernin IT Disallow: / User-agent: bigsur.ai Disallow: / User-agent: BizInformation Disallow: / User-agent: Black Hole Disallow: / User-agent: Blackboard Safeassign Disallow: / User-agent: Bloodhound Disallow: / User-agent: BlowFish/1.0 Disallow: / User-agent: BLP_bbot Disallow: / User-agent: bluematrix Disallow: / User-agent: BotALot Disallow: / User-agent: Brandwatch Disallow: / User-agent: Bravebot Disallow: / User-agent: Briefcase.news Disallow: / User-agent: Buck Disallow: / User-agent: BuiltBotTough Disallow: / User-agent: Bullseye/1.0 Disallow: / User-agent: BunnySlippers Disallow: / User-agent: CCBo Disallow: / User-agent: Cegbfeieh Disallow: / User-agent: Channel3Bot Disallow: / User-agent: ChatGLM-Spider Disallow: / User-agent: ChatGPT Agent Disallow: / Allow: /thema User-agent: CheeseBot Disallow: / User-agent: CherryPicker Disallow: / User-agent: CherryPickerElite/1.0 Disallow: / User-agent: CherryPickerSE/1.0 Disallow: / User-agent: CikisiBot Disallow: / User-agent: Cision Disallow: / User-agent: Claude-SearchBot Disallow: / User-agent: Claude-User Disallow: / User-agent: Cloudflare-AutoRAG Disallow: / User-agent: CloudVertexBot Disallow: / User-agent: coexel Disallow: / User-agent: cohere-training-data-crawler Disallow: / User-agent: Comtex News Network Disallow: / User-agent: ConveraCrawler Disallow: / User-agent: CopyRightCheck Disallow: / User-agent: Corporama Disallow: / User-agent: cosmos Disallow: / User-agent: Cotoyogi Disallow: / User-agent: CotoyogiBot Disallow: / User-agent: Crawl4AI Disallow: / User-agent: Crescent Disallow: / User-agent: Crescent Internet ToolPak HTTP OLE Control v.1.0 Disallow: / User-agent: cydralspider Disallow: / User-agent: D&B Hoovers Disallow: / User-agent: Data Expression Disallow: / User-agent: Data Observer Disallow: / User-agent: Dataminr Disallow: / User-agent: Datenbank Crawler Disallow: / User-agent: Dealogic Disallow: / User-agent: Devin Disallow: / User-agent: Digimind Disallow: / User-agent: DirectFN Disallow: / User-agent: DISCo Pump 3.1 Disallow: / User-agent: DittoSpyder Disallow: / User-agent: dotbot Disallow: / User-agent: Download Ninja Disallow: / User-agent: downloadexpress Disallow: / User-agent: DownloadNinja Disallow: / User-agent: DuckAssistBot Disallow: / User-agent: Dun & Bradstreet Disallow: / User-agent: Dun & Bradstreet - D&B ESG Intelligence Disallow: / User-agent: Dun & Bradstreet Data Marketplace Disallow: / User-agent: Eagle Alpha Disallow: / User-agent: ecoresearch Disallow: / User-agent: EDD Disallow: / User-agent: EmailCollector Disallow: / User-agent: EmailSiphon Disallow: / User-agent: EmailWolf Disallow: / User-agent: EroCrawler Disallow: / User-agent: eureka Disallow: / User-agent: Europresse Disallow: / User-agent: ExaBot Disallow: / User-agent: Explore Disallow: / User-agent: ExtractorPro Disallow: / User-agent: Factiva Disallow: / User-agent: Factset_spyderbot Disallow: / User-agent: Fasterfox Disallow: / User-agent: FeedCheck Disallow: / User-agent: FeedReader Disallow: / User-agent: Feedspot Disallow: / User-agent: Fetch Disallow: / User-agent: FirecrawlAgent Disallow: / User-agent: Fitch Solutions Disallow: / User-agent: Flamingo_SearchEngine Disallow: / User-agent: flipboard Disallow: / User-agent: Foobot Disallow: / User-agent: Founder Apabi Disallow: / User-agent: Freshbot Disallow: / User-agent: FriendlyCrawler Disallow: / User-agent: gammaSpider Disallow: / User-agent: gemini-deep-research Disallow: / Allow: /thema User-agent: Gnowit Disallow: / User-agent: GnowitNewsbot Disallow: / User-agent: Ground News Disallow: / User-agent: grub-client Disallow: / User-agent: Harvest/1.5 Disallow: / User-agent: heritrix Disallow: / User-agent: hloader Disallow: / User-agent: httplib Disallow: / User-agent: HTTrack Disallow: / User-agent: HTTrack 3.0 Disallow: / User-agent: humanlinks Disallow: / User-agent: ia_archiver Disallow: / User-agent: ia_archiver-web.archive.org Disallow: / User-agent: iAskBot Disallow: / User-agent: IAskBot Disallow: / User-agent: iaskspider Disallow: / User-agent: ICC-Crawler Disallow: / User-agent: Igentia Disallow: / User-agent: ImageSift Disallow: / User-agent: imageSpider Disallow: / User-agent: img2dataset Disallow: / User-agent: indexer Disallow: / User-agent: InfoNaviRobot Disallow: / User-agent: infoseek Disallow: / User-agent: InMédia Technologies Disallow: / User-agent: Innguma Disallow: / User-agent: Inoreader Disallow: / User-agent: ISI Emerging Markets Disallow: / User-agent: JennyBot Disallow: / User-agent: Jetbot Disallow: / User-agent: JikeSpider Disallow: / User-agent: k2spider Disallow: / User-agent: kagi-fetcher Disallow: / User-agent: Kangaroo Bot Disallow: / User-agent: Kantar Disallow: / User-agent: kbcrawl Disallow: / User-agent: Kenjin Spider Disallow: / User-agent: KlaviyoAIBot Disallow: / User-agent: Knowings Disallow: / User-agent: Koyfin Disallow: / User-agent: KunatoBot Disallow: / User-agent: KunatoCrawler Disallow: / User-agent: laion-huggingface-processor Disallow: / User-agent: larbin Disallow: / User-agent: Launchmetrics Disallow: / User-agent: LCC Disallow: / User-agent: leadbox Disallow: / User-agent: LexiBot Disallow: / User-agent: LexisNexis Disallow: / User-agent: Liana Disallow: / User-agent: libWeb/clsHTTP Disallow: / User-agent: libwww Disallow: / User-agent: LinerBot Disallow: / User-agent: LinkextractorPro Disallow: / User-agent: linkfluence Disallow: / User-agent: linko Disallow: / User-agent: LinkScan/8.1a Unix Disallow: / User-agent: LinkupBot Disallow: / User-agent: LinkWalker Disallow: / User-agent: lwp-trivial Disallow: / User-agent: lwp-trivial/1.34 Disallow: / User-agent: magpie-crawler Disallow: / User-agent: Make/production Disallow: / User-agent: manageo Disallow: / User-agent: MarketResearch.com Disallow: / User-agent: MarketWatch Disallow: / User-agent: MarketWise Disallow: / User-agent: Markit Digital Disallow: / User-agent: Mata Hari Disallow: / User-agent: mediacompil Disallow: / User-agent: Mediatoolkitbot Disallow: / User-agent: Meltwater Disallow: / User-agent: mention Disallow: / User-agent: meta-externalfetcher Disallow: / User-agent: Microsoft URL Control - 5.01.4511 Disallow: / User-agent: Microsoft URL Control - 6.00.8169 Disallow: / User-agent: MIIxpc Disallow: / User-agent: Mister PiX Disallow: / User-agent: Mistral.ai/crawler Disallow: / User-agent: MLBot Disallow: / User-agent: moduleQ Disallow: / User-agent: moget Disallow: / User-agent: moget/2.1 Disallow: / User-agent: MONITIO Disallow: / User-agent: MoodleBot Disallow: / User-agent: Moody's Disallow: / User-agent: Moreover Disallow: / User-agent: MORNINGSTAR Disallow: / User-agent: MSIECrawler Disallow: / User-agent: MuckRack Disallow: / User-agent: mytwip Disallow: / User-agent: Naverbot Disallow: / User-agent: NetAnts Disallow: / User-agent: NetAttache Disallow: / User-agent: netEstate Imprint Crawler Disallow: / User-agent: NetMechanic Disallow: / User-agent: Netvibes Disallow: / User-agent: news-api.org Disallow: / User-agent: newscan-online Disallow: / User-agent: Newslitbot Disallow: / User-agent: NewsNow Disallow: / User-agent: Newzbin Disallow: / User-agent: NICErsPRO Disallow: / User-agent: NPBot Disallow: / User-agent: ObjectsSearch Disallow: / User-agent: Onclusive Disallow: / User-agent: Openfind Disallow: / User-agent: OpenindexSpider Disallow: / User-agent: opinion-tracker Disallow: / User-agent: Opoint Disallow: / User-agent: Orbis Disallow: / User-agent: PanguBot Disallow: / User-agent: Paqlebot Disallow: / User-agent: Perplexity-User Disallow: / Allow: /thema User-agent: PhindBot Disallow: / User-agent: Pimptrain Disallow: / User-agent: Poggio-Citations Disallow: / User-agent: Press Monitor Europe Disallow: / User-agent: PressEngineBot D Disallow: / User-agent: PriberamBot Disallow: / User-agent: ProPowerBot/2.14 Disallow: / User-agent: ProRataIncBot Disallow: / User-agent: ProWebWalker Disallow: / User-agent: proxem Disallow: / User-agent: psbot Disallow: / User-agent: QQ Disallow: / User-agent: QualifiedBot Disallow: / User-agent: QuepasaCreep Disallow: / User-agent: QueryN Metasearch Disallow: / User-agent: QuoteMedia Disallow: / User-agent: Qwam content intelligence Disallow: / User-agent: QWAM CONTENT INTELLIGENCE Disallow: / User-agent: Qwam-content-intelligence Disallow: / User-agent: Qzone Disallow: / User-agent: RankurBot Disallow: / User-agent: Raven Disallow: / User-agent: RavenPack Disallow: / User-agent: readability.com Disallow: / User-agent: RepoMonkey Disallow: / User-agent: ReportLinker Disallow: / User-agent: Research & Markets Disallow: / User-agent: RMA Disallow: / User-agent: RyeBot Disallow: / User-agent: S&P Capital IQ Disallow: / User-agent: S&P Global Market Intelligence Disallow: / User-agent: S&P Global Marketplace Disallow: / User-agent: SBIntuitionsBot Disallow: / User-agent: scoop.it Disallow: / User-agent: scoopit
Changes recorded since June 2026.
No change recorded.