CrawlerPulse
CrawlerPulse / Europe / Spain / elpuntavui.cat

elpuntavui.cat

elpuntavui.cat · Spain · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Blocked
Anthropic
Blocked
Perplexity
Blocked
Meta
Allowed
Common Crawl
Blocked
ByteDance
Allowed
Apple
Blocked
Amazon
Blocked

robots.txt

As read on 8 Oct 2026, 2,282 bytes. 2 rule groups name AI crawlers; shown first.

User-Agent: ClaudeBot
User-Agent: Claude-User
User-Agent: Claude-SearchBot
User-Agent: GPTBot
User-Agent: GeminiBot
User-Agent: PerplexityBot
User-Agent: CCBot
User-Agent: YouBot
User-Agent: AllenAI Bot
user-Agent: Applebot-Extended
Disallow: /

User-Agent: AiHitBot
User-Agent: AlphaSeoBot
User-Agent: AlphaSeoBot-SA
User-Agent: AmazonBot
User-agent: barkrowler
User-Agent: BLEXBot
User-Agent: CareerBot
User-Agent: Cliqzbot
User-Agent: MegaIndex
User-Agent: Moreover
User-agent: OpenindexSpider
User-agent: OAI-SearchBot
User-Agent: Owlin
User-Agent: Owlin Bot
User-Agent: Owlin Bot v. 3.0
User-Agent: Paqlebot
User-Agent: QuerySeekerSpider
User-Agent: SEOkicks-Robot
User-Agent: Semrush
User-Agent: SemrushBot
User-Agent: Sogou
User-Agent: TkBot
User-Agent: Vagabondo
User-Agent: WBSearchBot
User-Agent: dotbot
User-Agent: grapeshot
User-Agent: iCjobs
User-Agent: linkdexbot
User-Agent: magpie-crawler
User-Agent: proximic
User-Agent: scrapy
User-Agent: scrapybot
User-Agent: sentibot
User-agent: AdIdxBot
User-agent: AhrefsBot
User-agent: BLP_bbot
User-agent: BLP_bbot/0.1
User-agent: Baiduspider
User-agent: Baiduspider+
User-agent: Baiduspider-image+
User-agent: BingPreview
User-agent: Exabot/3.0
User-agent: MJ12bot
User-agent: NaverBot
User-agent: SemrushBot
User-agent: Snapbot/1.0
User-agent: Yandex
User-agent: YandexBot
User-agent: Yeti
User-agent: YoudaoBot
User-agent: dtSearchSpider
User-agent: ichiro
User-agent: magpie-crawler
User-agent: moget
User-agent: sogou spider
User-agent: Screaming Frog SEO Spider
User-agent: trendictionbot0.5.0
User-agent: trendictionbot
Disallow: /

User-agent: Twitterbot
Disallow: /en-temps-real.html

User-Agent: trendkite-akashic-crawler
Disallow: /

User-agent: *
Disallow: /administrator/
Disallow: /cache/
Disallow: /component/epalogin
Disallow: /components/
Disallow: /images/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /libraries/
Disallow: /media2/
Disallow: /modules/
Disallow: /plugins/
Disallow: /templates2/
Disallow: /tmp/
Disallow: /xmlrpc/
Disallow: /cgi-bin/
Disallow: /serveirestringit.html
Disallow: /*.raw
Disallow: /edicio/
Disallow: /*/*.raw
Disallow: /*/*.raw?*
Disallow: *.jpg?ESBORRAM*
Disallow: /cercar
Disallow: /cercar.html

History

Changes recorded since June 2026.

No change recorded.

In context

Spain
34.1% block at least one AI crawler
Behind Cloudflare
Yes