CrawlerPulse
CrawlerPulse / Europe / Italy / vive-liberta.livejournal.com

Liberta

vive-liberta.livejournal.com · Italy · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 931 bytes. No rule names an AI crawler.

User-Agent: *
Allow: /data/rss/
Disallow: /*.html*mode=reply
Disallow: /*.html*replyto
Disallow: /data/foaf/
Disallow: /tag/
Disallow: /friendstimes
Disallow: /d0*
Disallow: /d1*
Disallow: /d2*
Disallow: /d3*
Disallow: /d4*
Disallow: /d5*
Disallow: /d6*
Disallow: /d7*
Disallow: /d8*
Disallow: /d9*
Disallow: /ratings/users
Disallow: *.html?
Disallow: /media
Disallow: /map
Disallow: /memories?

User-Agent: Mediapartners-Google*
Allow: /

User-Agent: Yandex
Allow: /
Disallow: /data/foaf/
Disallow: /ratings/users
Disallow: /photo
Disallow: *.html?
Disallow: /media
Disallow: /map
Disallow: /memories?
Clean-param: nojs&tag&mode&thread
Host: https://vive-liberta.livejournal.com
Crawl-delay: 100

User-Agent: GoogleBot
Allow: /
Disallow: /ratings/users
Disallow: /data/foaf/
Disallow: /photo
Disallow: *.html?
Disallow: /media
Disallow: /map
Disallow: /memories?

User-Agent: spbot
Disallow: /

User-agent: AhrefsBot
Disallow: /

History

Changes recorded since June 2026.

No change recorded.

In context

Italy
54.5% block at least one AI crawler
Behind Cloudflare
No