CrawlerPulse
CrawlerPulse / Europe / France / lepetitbuzz.fr

Le petit Buzz

lepetitbuzz.fr · France · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Partial
OpenAI
Partial
Anthropic
Partial
Perplexity
Partial
Meta
Partial
Common Crawl
Partial
ByteDance
Partial
Apple
Partial
Amazon
Partial

robots.txt

As read on 8 Oct 2026, 1,015 bytes. No rule names an AI crawler.

User-agent: *
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content/plugins
Disallow: /wp-content/cache
Disallow: /trackback
Disallow: /feed
Disallow: /comments
Disallow: /category/*/*
Disallow: */trackback
Disallow: */feed
Disallow: */comments
Disallow: /*.pdf$
Disallow: /*?*
Disallow: /*?
Disallow: /wp-login.php
Allow: /wp-content/uploads

User-agent: Googlebot
Disallow: /*.php$
Disallow: /*.inc$
Disallow: /*.gz$
Disallow: /*.pdf$

User-agent: Googlebot-Image
Disallow:
Allow: /*

User-agent: Mediapartners-Google*
Disallow:
Allow: /*

User-agent: AhrefsSiteAudit
Allow: /*

User-agent: AhrefsBot
Allow: /*

History

Changes recorded since June 2026.

No change recorded.

In context

France
42.3% block at least one AI crawler
Behind Cloudflare
No