elcorreoweb.es · Spain · Prensa Ibérica · measured 8 Oct 2026
As read on 8 Oct 2026, 2,492 bytes. 9 rule groups name AI crawlers; shown first.
User-agent: GPTBot Allow: /*-bc- Allow: /ideas/ Disallow: / User-agent: OAI-SearchBot Allow: /*-bc- Allow: /ideas/ Disallow: / User-agent: ChatGPT-User Allow: /*-bc- Allow: /ideas/ Disallow: / User-agent: PerplexityBot Allow: /*-bc- Allow: /ideas/ Disallow: / User-agent: ClaudeBot Allow: /*-bc- Allow: /ideas/ Disallow: / User-agent: Claude-Web Allow: /*-bc- Allow: /ideas/ Disallow: / User-agent: Meta-ExternalAgent Disallow: / User-agent: Google-Extended Allow: / User-agent: Google-CloudVertexBot Allow: / User-agent: 008 Disallow: /tic User-agent: UnisterBot Disallow: / User-agent: JikeSpider Disallow: / User-agent: Zealbot Disallow: / User-agent: WebStripper Disallow: / User-agent: WebCopier Disallow: / User-agent: Fetch Disallow: / User-agent: Teleport Disallow: / User-agent: HTTrack Disallow: / User-agent: magpie-crawler Disallow: / User-agent: MJ12bot Disallow: / User-agent: DotBot Disallow: / User-agent: spbot Disallow: / User-agent: BLEXBot Disallow: / User-agent: BUbiNG Disallow: / User-agent: Yandex Disallow: / User-agent: Baiduspider Disallow: / User-agent: Perplexity-User Allow: /*-bc- Allow: /ideas/ Disallow: / User-agent: Claude-User Allow: /*-bc- Allow: /ideas/ Disallow: / User-agent: Claude-SearchBot Allow: /*-bc- Allow: /ideas/ Disallow: / User-agent: AmazonAdBot Allow: / User-Agent: * Allow: /*-bc- Allow: /*-sh/index.html$ Allow: /*-rm/index.html$ Disallow: /deportes/futbol/ Disallow: /deportes/baloncesto/ Disallow: /economia/activos/ Disallow: /clip/ Disallow: /fonts/ Disallow: /ocio/hosteleria/hoteles/ Disallow: /ocio/planes/planes/ Disallow: /ocio/gastronomia/vinos/mejores/ Disallow: /ocio/gastronomia/cocteles/ Disallow: /ocio/hosteleria/hoteles/ Disallow: /ocio/hosteleria/restaurantes/ Disallow: /olimpiadas/disciplinas/ Disallow: /olimpiadas/calendario/ Disallow: /olimpiadas/medallero/ Disallow: /olimpiadas/resultados/ Disallow: /hemeroteca/ Disallow: /buscador/ Disallow: /*?p= Disallow: /*CANONICAL_URL Disallow: /elecciones/municipales/28m/42ds/fonts/ Disallow: /elecciones/municipales/28m/ Disallow: /tags/$ Disallow: /tags/letra/ User-agent: Twitterbot Allow: / User-agent: facebookexternalhit Allow: / Sitemap: https://www.elcorreoweb.es/sitemapNews.xml Sitemap: https://www.elcorreoweb.es/sitemapMonth.xml Sitemap: https://www.elcorreoweb.es/sitemapSection.xml
Changes recorded since June 2026.
No change recorded.