CrawlerPulse
CrawlerPulse / Africa / Zambia / mwebantu.com

mwebantu.com

mwebantu.com · Zambia · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Blocked
OpenAI
Blocked
Anthropic
Blocked
Perplexity
Blocked
Meta
Blocked
Common Crawl
Blocked
ByteDance
Blocked
Apple
Blocked
Amazon
Blocked

robots.txt

As read on 8 Oct 2026, 461 bytes. 1 rule group name AI crawlers; shown first.

User-agent: FacebookBot
Allow: /
Crawler-delay: 10

User-agent: *
Disallow: /

User-agent: GoogleBot
Allow: /
Crawler-delay: 10

User-agent: FacebookExternalHit
Allow: /
Crawler-delay: 10

User-agent: BingBot
Allow: /
Crawler-delay: 10

User-agent: YahooBot
Allow: /
Crawler-delay: 10

User-agent: SemrushBot
Allow: /
Crawler-delay: 10

User-agent: MJ12bot
Allow: /
Crawler-delay: 10
Sitemap: https://www.mwebantu.com/sitemap.xml

History

Changes recorded since June 2026.

No change recorded.

In context

Zambia
3.1% block at least one AI crawler
Behind Cloudflare
Yes