CrawlerPulse
CrawlerPulse / Europe / Spain / europapress.es

Europa Press

europapress.es · Spain · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 1,014 bytes. No rule names an AI crawler.

User-agent: *
Disallow: /abonados/
Disallow: /europa/
Disallow: /europa2001/
Disallow: /europa2003/
Disallow: /boletin-joven-00135/
Disallow: /rsc-00195/
Disallow: /enviarnoticia.aspx
Disallow: /enviatunoticia.aspx
Disallow: /ws/AltaBoletin.ashx
Disallow: /buscador/
Sitemap: https://www.europapress.es/sitemap.xml
Sitemap: https://www.europapress.es/sitemap_index/

User-agent: Yandex
Disallow: /

User-agent: Mail.Ru
Disallow: /

User-agent: Freshbot
Disallow: /

User-agent: IstellaBot
Disallow: /

User-agent: AhrefsBot
Disallow: /

User-agent: trendictionbot
Disallow: /

User-agent: sentibot
Disallow: /

User-agent: grapeshot
Crawl-Delay: 5

User-agent: heritrix
Crawl-Delay: 5

User-agent: MJ12bot
Crawl-Delay: 5

User-agent: um-IC
Crawl-Delay: 1

User-agent: SiteAuditBot
Disallow: /

User-agent: Slurp
Crawl-Delay: 10

User-agent: dotbot
Disallow: /

User-agent: SeekportBot
Crawl-delay: 2

User-agent: barkrowler
Disallow: /

History

Changes recorded since June 2026.

No change recorded.

In context

Spain
34.1% block at least one AI crawler
Behind Cloudflare
No