CrawlerPulse
CrawlerPulse / Europe / Spain / eldesmarque.com

El Desmarque

eldesmarque.com · Spain · Grupo Audiovisual Mediaset España Comunicación, S.A.U. · measured 4 Aug 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 4 Aug 2026, 635 bytes. 1 rule group name AI crawlers; shown first.

User-agent: FacebookBot
Crawl-delay: 10

User-agent: *
Disallow: /buscador/*?text=*
Disallow: /mdswebservice/
Disallow: /mdsvideo/
Disallow: /mdsads/
Disallow: /stats.html
Disallow: /api/boards
Disallow: /tags/*?text=*
Disallow: /autores/*?text=*
Disallow: /personajes/*?text=*
Disallow: /20n/

User-agent: facebookexternalhit
Crawl-delay: 10

User-agent: Twitterbot/1.0
Crawl-delay: 10

User-agent: msnbot
Disallow: *.shtml

User-agent: yaanibot
Disallow: /
Sitemap: https://www.eldesmarque.com/sitemap_index.xml
Sitemap: https://www.eldesmarque.com/sitemap_ampstories.xml
Sitemap: https://www.eldesmarque.com/sitemap_google_news.xml

History

Changes recorded since June 2026.

No change recorded.

In context

Spain
34.1% block at least one AI crawler
Behind Cloudflare
No
Same group
cuatro.com, niusdiario.es, telecinco.es