CrawlerPulse
CrawlerPulse / Europe / Spain / diariojaen.es

Diario Jaén

diariojaen.es · Spain · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 977 bytes. No rule names an AI crawler.

sitemap: https://www.diariojaen.es/sitemap.xml
sitemap: https://www.diariojaen.es/sitemapforgoogle.xml
sitemap: https://www.diariojaen.es/megasitemap.xml

User-agent: *
Disallow: /news-portlet/metalocator/
Disallow: /news-portlet/html/teaser-viewer-portlet/teaser_page.jsp
Disallow: /news-portlet/html/teaser-viewer-portlet/teaser_filter.jsp
Disallow: /news-portlet/filterteaser/
Disallow: /news-portlet/getfilteropts/
Disallow: /tracking-portlet/html/ranking-viewer/ranking_details.jsp
Disallow: /user-portlet/login-with/
Disallow: /user-portlet/edit-user-profile/
Disallow: /user-portlet/reset-credentials/
Disallow: /user-portlet/confirm-email/
Disallow: /user-portlet/refreshuserentitlements/
Disallow: /user-portlet/getEntitlements/
Disallow: /group/
Disallow: /user/
Disallow: /web/
Disallow: /image/
Disallow: /busqueda/
Disallow: /usuarios/recuperar-contrasena/
Disallow: /usuarios/perfil/
Disallow: /usuarios/registrate/

History

Changes recorded since June 2026.

No change recorded.

In context

Spain
34.1% block at least one AI crawler
Behind Cloudflare
No