CrawlerPulse
CrawlerPulse / Europe / Spain / sevilla.abc.es

ABC Sevilla

sevilla.abc.es · Spain · Vocento · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Blocked
Anthropic
Blocked
Perplexity
Blocked
Meta
Blocked
Common Crawl
Blocked
ByteDance
Blocked
Apple
Blocked
Amazon
Blocked

robots.txt

As read on 8 Oct 2026, 5,231 bytes. 2 rule groups name AI crawlers; shown first.

User-agent: Omgili
Disallow: /

User-agent: AI2Bot
User-agent: AI2Bot-DeepResearchEval
User-agent: Amazonbot
User-agent: Amzn-SearchBot
User-agent: Amzn-User
User-agent: Applebot-Extended
User-agent: archive.org_bot
User-agent: Bytespider
User-agent: CCBot
User-agent: ChatGPT-User
User-agent: Claude-SearchBot
User-agent: Claude-User
User-agent: ClaudeBot
User-agent: cohere-ai
User-agent: cohere-training-data-crawler
User-agent: Cotoyogi
User-agent: DuckAssistBot
User-agent: GPTBot
User-agent: ia_archiver
User-agent: ia_archiver-web.archive.org
User-agent: Kangaroo Bot
User-agent: meta-externalagent
User-agent: meta-externalfetcher
User-agent: meta-webindexer
User-agent: MistralAI-User
User-agent: OAI-SearchBot
User-agent: Perplexity-User
User-agent: PerplexityBot
User-agent: YouBot
Allow: /favorito
Allow: /contentfactory
Allow: /content-local
Disallow: /

User-Agent: *
Disallow: /mobility/wp-admin/
Disallow: /tmp/
Disallow: /temas/
Disallow: /trailers/
Disallow: /temas-personajes/
Disallow: /temas-lugares/
Disallow: /temas-acontecimientos/
Disallow: /temas-instituciones/
Disallow: /temas-obras/
Disallow: /asistente-ia/
Disallow: /elecciones/20n-2011/
Disallow: /20n-2011/
Disallow: /?cuarto=
Disallow: /pasionensevilla/wp-admin/
Disallow: /pasionensevilla/comunes/
Disallow: /pasionensevilla/templates/
Disallow: /pasionensevilla/dax/
Disallow: /pasionensevilla/m/
Disallow: /pasionensevilla/*.swf$
Disallow: /pasionensevilla/*.tif$
Disallow: /pasionensevilla/*.mp3$
Disallow: /pasionensevilla/*.flv$
Disallow: /pasionensevilla/*.mp4$
Disallow: /pasionensevilla/*.avi$
Disallow: /pasionensevilla/*.inc$
Disallow: /pasionensevilla/*.ics$
Disallow: /tusrecetas/administrator/
Disallow: /tusrecetas/cgi-bin/
Disallow: /tusrecetas/cache/
Disallow: /tusrecetas/comunes/
Disallow: /tusrecetas/includes/
Disallow: /tusrecetas/languages/
Disallow: /tusrecetas/libraries/
Disallow: /tusrecetas/logs/
Disallow: /tusrecetas/media/
Disallow: /tusrecetas/modules/
Disallow: /tusrecetas/plugins/
Disallow: /tusrecetas/templates/
Disallow: /tusrecetas/tmp/
Disallow: /tusrecetas/xmlrpc/
Disallow: /tusrecetas/lacocina2/
Disallow: /tusrecetas/lacocina/
Disallow: /*.swf$
Disallow: /*.tif$
Disallow: /*.flv$
Disallow: /*.mp4$
Disallow: /*.avi$
Disallow: /*.inc$
Disallow: /*.ics$
Disallow: /recortes/
Disallow: /recortes/*
Disallow: *//.html
Disallow: /temas-*%7C%7Chttp://resizer
Disallow: */home$
Disallow: /xml-sindicacion/
Disallow: /sindicacion/
Disallow: /c1m/menu/time/
Disallow: /4900/webm.ABCSEVILLA/
Disallow: /gl-d/
Disallow: /*/gl-d$
Disallow: /ht-d/
Disallow: /*/ht-d$
Disallow: /bb-d/
Disallow: /*/bb-d$
Disallow: /deportes/futbol/directos/*/2010-2011/*
Disallow: /deportes/futbol/directos/*/2011-2012/*
Disallow: /deportes/futbol/directos/*/2012-2013/*
Disallow: /deportes/futbol/directos/*/2013-2014/*
Disallow: /deportes/futbol/directos/*/2014-2015/*
Disallow: /deportes/futbol/directos/*/2015-2016/*
Disallow: /hemeroteca/historico-18-06-2004/sevilla/Sevilla/el-fiscal-dice-que-ex-empleados-de-gallardo-sebaste-eran-sus-hombres-de-paja_9622091130558.html
Disallow: /hemeroteca/historico-16-06-2004/sevilla/Sevilla/gallardo-atribuye-las-querellas-contra-el-a-un-asedio-de-ex-socios-y-ex-empleados_9622059183706.html
Disallow: /gurme/*/busqueda/?
Allow: /*-nt.html$
Allow: /*-di.html$
Allow: /*-vid.html$
Allow: /*-ga.html$
Disallow: /*-ft.html$
Disallow: /*-aud.html$
Disallow: /*-ntps.html$
Disallow: */aggregate*
Disallow: /includes_comun/*
Disallow: /*?cartelera_cine
Disallow: */wp-content/
Disallow: */wp-includes/
Disallow: /acd/
Allow: /*/2014*/
Allow: /2013*/
Allow: /*/2013*/
Allow: /2012*/
Allow: /2011*/
Allow: /2010*/
Allow: /2009*/
Allow: /2008*/
Allow: /2007*/
Allow: /2006*/
Allow: /2005*/
Allow: /2004*/
Allow: /2003*/
Allow: /2002*/
Allow: /2001*/
Allow: /2000*/
Allow: /hemeroteca/index.aspx
Allow: /hemeroteca/*/sevilla/
Allow: /hemeroteca/*/andalucia/
Allow: /*/sevilla-*/
Allow: /*/andalucia-*/
Allow: /*/nacional-sevilla-*/
Allow: /*/nacional-andalucia-*/
Allow: /sevilla
Allow: /andalucia
Allow: /sevilla
Allow: /provincia
Allow: /en-detalle
Allow: /*/provincia/
Allow: /*/en-detalle/
Allow: /*/toros/*

User-agent: psbot
Disallow: /

User-agent: Meltwater
Disallow: /

User-agent: ia_archiver
Disallow: /

User-agent: Fasterfox
Disallow: /

User-agent: UbiCrawler
Disallow: /

User-agent: DOC
Disallow: /

User-agent: Zao
Disallow: /

User-agent: sitecheck.internetseer.com
Disallow: /

User-agent: Zealbot
Disallow: /

User-agent: MSIECrawler
Disallow: /

User-agent: SiteSnagger
Disallow: /

User-agent: WebStripper
Disallow: /

User-agent: WebCopier
Disallow: /

User-agent: Fetch
Disallow: /

User-agent: Offline Explorer
Disallow: /

User-agent: Teleport
Disallow: /

User-agent: TeleportPro
Disallow: /

User-agent: WebZip
Disallow: /

User-agent: linko
Disallow: /

User-agent: HTTrack
Disallow: /

User-agent: Microsoft.URL.Control
Disallow: /

User-agent: Xenu
Disallow: /

User-agent: larbin
Disallow: /

User-agent: libwww
Disallow: /

User-agent: ZyBORG
Disallow: /

User-agent: Download Ninja
Disallow: /

User-agent: NewsNow
Disallow: /

User-agent: WebVac
Disallow: /

User-agent: 008
Disallow: /

User-agent: Twitterbot
Allow: *

History

Changes recorded since June 2026.

No change recorded.

In context