rnz.co.nz · New Zealand · Radio New Zealand · measured 8 Oct 2026
As read on 8 Oct 2026, 1,678 bytes. 19 rule groups name AI crawlers; shown first.
User-Agent: Amazonbot Disallow: / User-Agent: anthropic-ai Disallow: / User-Agent: Applebot-Extended Disallow: / User-Agent: Bytespider Disallow: / User-Agent: CCBot Disallow: / User-Agent: ChatGPT-User Disallow: / User-Agent: ClaudeBot Disallow: / User-Agent: Claude-Web Disallow: / User-Agent: cohere-ai Disallow: / User-Agent: Diffbot Disallow: / User-Agent: Google-CloudVertexBot Disallow: / User-Agent: Google-Extended Disallow: / User-Agent: GPTBot Disallow: / User-Agent: MistralAI-User Disallow: / User-Agent: OAI-SearchBot Disallow: / User-Agent: omgili Disallow: / User-Agent: omgilibot Disallow: / User-Agent: PerplexityBot Disallow: / User-Agent: YouBot Disallow: / User-agent: * Disallow: /admin/ Sitemap: https://www.rnz.co.nz/sitemap/sitemap.xml.gz Crawl-delay: 7 User-agent: psbot Disallow: / User-agent: magpie-crawler Disallow: / User-agent: turnitinbot Disallow: / User-Agent: * Allow: / User-Agent: Claude-SearchBot Disallow: / User-Agent: DuckAssistBot Disallow: / User-Agent: FriendlyCrawler Disallow: / User-Agent: Gemini-Deep-Research Disallow: / User-Agent: GoogleOther Disallow: / User-Agent: ICC-Crawler Disallow: / User-Agent: img2dataset Disallow: / User-Agent: peer39_crawler Disallow: / User-Agent: peer39_crawler/1.0 Disallow: / User-Agent: Perplexity-User Disallow: / User-Agent: Webz.io Disallow: / User-Agent: facebookexternalhit Allow: / User-agent: Applebot Crawl-delay: 10 User-agent: bingbot User-agent: msnbot User-agent: Yandex Crawl-delay: 20 User-agent: Wget Crawl-delay: 10
Changes recorded since June 2026.
No change recorded.