CrawlerPulse
CrawlerPulse / Africa / South Africa / bhekisisa.org

bhekisisa.org

bhekisisa.org · South Africa · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 474 bytes. No rule names an AI crawler.

User-agent: *
Disallow: /2024/
Allow: /2024/*/
Disallow: /2023/
Allow: /2023/*/
Disallow: /2022/
Allow: /2022/*/
Disallow: /2021/
Allow: /2021/*/
Disallow: /2020/
Allow: /2020/*/
Disallow: /2019/
Allow: /2019/*/
Disallow: /2018/
Allow: /2018/*/
Disallow: /2017/
Allow: /2017/*/
Disallow: /2016/
Allow: /2016/*/
Disallow: /2015/
Allow: /2015/*/
Disallow: /2014/
Allow: /2014/*/
Disallow: /2013/
Allow: /2013/*/
Disallow: /home/page/
Disallow: /page/

History

Changes recorded since June 2026.

No change recorded.

In context

South Africa
22.1% block at least one AI crawler
Behind Cloudflare
Yes