diarimes.com · Spain · Segre · measured 8 Oct 2026
As read on 8 Oct 2026, 3,802 bytes. 1 rule group name AI crawlers; shown first.
User-agent: anthropic-ai User-agent: AI2Bot User-agent: Ai2Bot-Dolma User-agent: AmazonBot User-agent: ApifyBot User-agent: ApifyWebsiteContentCrawler User-agent: Applebot-Extended User-agent: Bytespider User-agent: CCBot User-agent: ChatGLM-Spider User-agent: ClaudeBot User-agent: cohere-ai User-agent: cohere-training-data-crawler User-agent: Cotoyogi User-agent: Crawl4AI User-agent: Datenbank Crawler User-agent: DeepSeekBot User-agent: Diffbot User-agent: FacebookBot User-agent: FirecrawlAgent User-agent: iAskBot User-agent: iaskspider User-agent: ICC-Crawler User-agent: Kangaroo Bot User-agent: KunatoCrawler User-agent: laion-huggingface-processor User-agent: LCC User-agent: meta-externalagent User-agent: netEstate Imprint Crawler User-agent: omgili User-agent: omgilibot User-agent: PanguBot User-agent: SBIntuitionsBot User-agent: Spider User-agent: TavilyBot User-agent: Timpibot User-agent: VelenPublicWebCrawler User-agent: webzio-extended User-agent: WRTNBot User-agent: 008 User-agent: AddThis.com User-agent: admantx User-agent: BDCbot User-agent: Bender User-agent: BIXOCRAWLER User-agent: bl.uk_lddc_bot User-agent: BLEXBot User-agent: BUbiNG User-agent: ChatGPT-User User-agent: Cliqzbot User-agent: CNCDialer User-agent: crawler4j User-agent: CrystalSemanticsBot User-Agent: CyberAlert User-agent: DigExt User-agent: discobot User-agent: discoverybot User-agent: dloader User-agent: dloader(NaverRobot) User-agent: DOC User-agent: dotbot User-agent: Download Ninja User-agent: DTS Agent User-agent: Exabot User-agent: Ezooms User-Agent: FairShare User-agent: Fetch User-agent: Flamingo_SearchEngine User-agent: Freshbot User-agent: Genieo User-agent: Gigabot User-agent: GPTBot User-agent: grub-client User-agent: Heritrix User-agent: heritrix/3.3.0 User-agent: HTTrack User-agent: ia_archiver User-agent: integromedb User-agent: IstellaBot User-agent: JikeSpider User-agent: Jyxobot User-agent: k2spider User-agent: Kimengi User-agent: Kimengi/nineconnections.com User-agent: larbin User-agent: LexxeBot/1.0 User-agent: libwww User-agent: linko User-agent: Livelapbot User-agent: magpie-crawler User-agent: Mail.Ru User-agent: Maxthon User-agent: MetaURI User-agent: Microsoft.URL.Control User-agent: MJ12bot User-agent: Moreover User-agent: Moreoverbot User-agent: MSIECrawler User-agent: nabot User-agent: NaverBot User-agent: NerdByNature.Bot User-agent: netEstate NE Crawler User-agent: NetSeer crawler User-agent: Newscan User-agent: NextGenSearchBot User-agent: NPBot User-agent: nutch User-agent: Offline Explorer User-agent: omgilibot User-agent: Orthogaffe User-agent: PiplBot User-agent: Pixray-Seeker User-agent: proximic User-agent: psbot User-agent: QuerySeekerSpider User-agent: rogerbot User-agent: seokicks User-agent: SEOkicks-Robot User-agent: SiteBot User-agent: SiteBot/0.1 User-agent: sitecheck.internetseer.com User-agent: SiteSnagger User-agent: Slurp User-agent: sogou User-agent: Sosospider User-agent: spbot User-agent: Spinn3r User-agent: Teleport User-agent: TeleportPro User-Agent: trendictionbot User-agent: trovitBot User-agent: TurnitinBot User-agent: UbiCrawler User-agent: uMBot-LN User-agent: UnisterBot User-agent: UniversalFeedParser User-agent: WBSearchBot User-agent: WebCopier User-agent: WebReaper User-agent: WebStripper User-agent: WebZIP User-agent: WeSEE:Search User-agent: wget User-agent: Wotbot User-agent: wotbox User-agent: Xenu User-agent: Yandex User-agent: Yasni User-agent: Zao User-agent: Zealbot User-agent: ZyBORG User-agent: Claude-web Disallow: / User-agent: * Disallow: /search Disallow: /es/search Disallow: /ca/search Allow: / Sitemap: https://www.diarimes.com/sitemap-index.xml Sitemap: https://www.diarimes.com/sitemap-google-news-ca.xml Sitemap: https://www.diarimes.com/sitemap-google-news-es.xml
Changes recorded since June 2026.
No change recorded.