CrawlerPulse
CrawlerPulse / Europe / Spain / cadenaser.com

Cadena SER

cadenaser.com · Spain · Sociedad Española de Radiodifusión · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Blocked
Perplexity
Blocked
Meta
Allowed
Common Crawl
Blocked
ByteDance
Blocked
Apple
Allowed
Amazon
Blocked

robots.txt

As read on 8 Oct 2026, 5,686 bytes. 8 rule groups name AI crawlers; shown first.

User-agent: PetalBot
Disallow: /

User-agent: Amazonbot
Disallow: /

User-agent: Bytespider
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: CCBot
Disallow: /

User-agent: DeepSeekBot
Disallow: /

User-agent: MistralAI-User
Disallow: /

User-agent: PerplexityBot
Disallow: /

User-agent: bedrock-agentcore-browser
Disallow: /

User-agent: Amazon-Bot
Disallow: /

User-agent: AmazonProductDiscoveryBot
Disallow: /

User-agent: Amzn-SearchBot
Disallow: /

User-agent: Amzn-User
Disallow: /

User-agent: Arquivo-web-crawler
Disallow: /

User-agent: bnf.fr_bot
Disallow: /

User-agent: Claude-Extension
Disallow: /

User-agent: Claude-User
Disallow: /

User-agent: Claude-SearchBot
Disallow: /

User-agent: Comet
Disallow: /

User-agent: CloudFlare-AlwaysOnline
Disallow: /

User-agent: Cloudflare-Healthchecks
Disallow: /

User-agent: CloudflareBrowserRendering
Disallow: /

User-agent: Cloudflare-Radar
Disallow: /

User-agent: DuckAssistBot
Disallow: /

User-agent: archive.org_bot
Disallow: /

User-agent: ia_archiver
Disallow: /

User-agent: Moreover
Disallow: /

User-agent: Adbeat
Disallow: /

User-agent: Argus
Disallow: /

User-agent: Baiduspider
Disallow: /

User-agent: Baiduspider-image
Disallow: /

User-agent: Bitdefender
Disallow: /

User-agent: Blogtrottr
Disallow: /

User-agent: BloombergBot
Disallow: /

User-agent: BoardReader
Disallow: /

User-agent: checkmy.ws
Disallow: /

User-agent: Clickagy
Disallow: /

User-agent: CookieBot
Disallow: /

User-agent: cXensebot
Disallow: /

User-agent: DataObserver
Disallow: /

User-agent: DomainStatsBot
Disallow: /

User-agent: Embedly
Disallow: /

User-agent: Feedspotbot
Disallow: /

User-agent: FlipboardProxy
Disallow: /

User-agent: Getintent
Disallow: /

User-agent: Google-Apps-Script
Disallow: /

User-agent: GoogleDrive
Disallow: /

User-agent: Grapevine6
Disallow: /

User-agent: Gumgum
Disallow: /

User-agent: HaloBot
Disallow: /

User-agent: HubSpot Crawler
Disallow: /

User-agent: IbouBot
Disallow: /

User-agent: Inout
Disallow: /

User-agent: LinerBot
Disallow: /

User-agent: MJ12bot
Disallow: /

User-agent: Manus
Disallow: /

User-agent: Meltwater
Disallow: /

User-agent: Metajob
Disallow: /

User-agent: Muckrack
Disallow: /

User-agent: NetSeer
Disallow: /

User-agent: Netvibes
Disallow: /

User-agent: Onyphe
Disallow: /

User-agent: OpenClaw
Disallow: /

User-agent: Opoint
Disallow: /

User-agent: OutbrainBot
Disallow: /

User-agent: PlumAnalytics
Disallow: /

User-agent: ProSitemaps
Disallow: /

User-agent: Perplexity-User
Disallow: /

User-agent: proximic
Disallow: /

User-agent: PubMatic
Disallow: /

User-agent: Quantcastbot
Disallow: /

User-agent: SearchConcepts
Disallow: /

User-agent: Semantic-Visions
Disallow: /

User-agent: SentiBot
Disallow: /

User-agent: Sindup
Disallow: /

User-agent: Sirdata
Disallow: /

User-agent: SmartologyBot
Disallow: /

User-agent: Superfeedr
Disallow: /

User-agent: surlybot
Disallow: /

User-agent: Synaptive
Disallow: /

User-agent: TinEye
Disallow: /

User-agent: Trendiction
Disallow: /

User-agent: TurnitinBot
Disallow: /

User-agent: Ubermetrics
Disallow: /

User-agent: Vebidoo
Disallow: /

User-agent: Webcorp
Disallow: /

User-agent: Weborama
Disallow: /

User-agent: WebPageTest
Disallow: /

User-agent: YandexAccessibilityBot
Disallow: /

User-agent: YandexDirect
Disallow: /

User-agent: YandexDirectDyn
Disallow: /

User-agent: YaDirectFetcher
Disallow: /

User-agent: YandexFavicons
Disallow: /

User-agent: YandexMetrika
Disallow: /

User-agent: YandexMobileBot
Disallow: /

User-agent: YandexNews
Disallow: /

User-agent: YandexRCA
Disallow: /

User-agent: YandexRenderResourcesBot
Disallow: /

User-agent: YandexUserproxy
Disallow: /

User-agent: *
Disallow: /*core_dmena

User-agent: *
Disallow: /pruebas/
Disallow: /publicidad/
Disallow: /notificarelacionadas
Disallow: /*.swf$
Disallow: /eskupTSUpdate
Disallow: /ThreadeskupSimple
Disallow: /OuteskupSimple
Disallow: /Ineskup
Disallow: /Outeskup
Disallow: /OuteskupSimpleMustache
Disallow: /Msgeskup
Disallow: /Profileeskup
Disallow: /Threadeskup
Disallow: /pxlctl.gif
Disallow: /pxlctl2.gif
Disallow: /includes/
Disallow: /especial/juegos-olimpicos/
Disallow: /pf/api/
Disallow: /tag//*
Disallow: */Aes/$
Disallow: */Ves/$
Disallow: /configuracion_externa/
Disallow: /drive-and-listen/
Disallow: /estaticos/recursosgraficos/responsive/especiales/amazon-interno/*
Disallow: /buscar/
Disallow: */buscador/
Disallow: /tag/fecha/
Disallow: /tag/c/ac0867c1271c39d492e04846c569c641/
Disallow: /tag/c/e60c1782635df88638703dc982a2f2cd/
Disallow: /tag/c/ac41b7d7a6e69ee6bd882135750242b8/
Disallow: /tag/c/2e4a1433dc59d21946b58238cc94e7ca/
Disallow: /hemeroteca/*
Disallow: /tag/
Allow: /tag/*/a/
Allow: /tag/*/b/
Allow: /tag/*/c/
Allow: /tag/*/a
Allow: /tag/*/b
Allow: /tag/*/c
Disallow: /autor/francisco_jose_pedrosa_rodriguez/a/*/
Allow: /autor/francisco_jose_pedrosa_rodriguez/a/
Disallow: /castillalamancha/2022/07/22/mister-toledo-y-su-gemelo-pillados-tras-liderar-una-red-estafadora-a-amazon-ser-toledo/
Disallow: /ser/2015/06/23/tribunales/1435079533_246872.html
Disallow: /ser/2016/01/07/tribunales/1452170057_978379.html
Disallow: /emisora/2016/05/06/radio_madrid/1462535866_996722.html
Disallow: /audio/20050624csrcsrnac_1.Aes
Disallow: /ser/2005/06/24/audios/1119570611_660215.html
Disallow: /nacional/2024/11/18/ramon-cugat-el-problema-con-la-lesion-de-rodilla-de-ansu-fati-no-es-la-operacion-es-la-recuperacion-que-se-hizo-cadena-ser/

History

Changes recorded since June 2026.

No change recorded.

In context

Spain
34.1% block at least one AI crawler
Behind Cloudflare
No