stuff.co.za · South Africa · measured 8 Oct 2026
As read on 8 Oct 2026, 1,734 bytes. 2 rule groups name AI crawlers; shown first.
User-agent: Amazonbot Disallow: / User-agent: Bytespider Disallow: / User-agent: * Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: /wp-content/plugins/ Disallow: /wp-config.php Disallow: /wp-load.php Disallow: /wp-cron.php Disallow: /wp-links-opml.php Disallow: /wp-mail.php Disallow: /wp-trackback.php Disallow: /wp-activate.php Disallow: /wp-signup.php Disallow: /wp-comments-post.php Disallow: /xmlrpc.php Disallow: /readme.html Disallow: /license.txt Disallow: /*?s= Disallow: /*&s= Disallow: /search/ Disallow: /*?preview=true Disallow: /*?replytocom= Disallow: /*/feed/ Disallow: /tag/*/feed/ Disallow: /category/*/feed/ Disallow: /author/*/feed/ Disallow: /*/trackback/ Disallow: /*.sql$ Disallow: /*.log$ Disallow: /*.bak$ Disallow: /*?page= Disallow: /*?paged= User-agent: Googlebot Allow: /wp-admin/admin-ajax.php User-agent: Bingbot Allow: /wp-admin/admin-ajax.php User-agent: SemrushBot Crawl-delay: 86400 User-agent: AhrefsBot Crawl-delay: 86400 User-agent: MJ12bot Crawl-delay: 86400 Sitemap: https://stuff.co.za/news-sitemap.xml Sitemap: https://stuff.co.za/sitemap_index.xml
Changes recorded since June 2026.
No change recorded.