CrawlerPulse
CrawlerPulse / Africa / Senegal / jotaay.net

jotaay.net

jotaay.net · Senegal · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 610 bytes. No rule names an AI crawler.

User-agent: *
Disallow: /*print=1$
Disallow: /*voir_commentaire=
Disallow: /*preaction=view_nl
Disallow: /*id_form=
Disallow: /*numform=
Disallow: /admin/
Allow: /docs/*.js
Allow: /docs/*.css
Allow: /docs/public/
Allow: /docs/img/
Allow: /docs/IMG/
Allow: /docs/js/
Allow: /docs/JS/
Allow: /docs/css/
Allow: /docs/CSS/
Allow: /docs/style/
Allow: /docs/javascript/
Allow: /docs/lib/
Disallow: /docs/
Disallow: /download/
Disallow: /send/
Disallow: /submit/
Disallow: /subscription/
Disallow: /members/
Disallow: /social/
Disallow: /forms/
Disallow: /dir/
Disallow: /sv/
Disallow: /svideo/
Disallow: /svideossd/

History

Changes recorded since June 2026.

No change recorded.

In context

Senegal
4.2% block at least one AI crawler
Behind Cloudflare
No