tradewindsnews.com · Norway · measured 9 Oct 2026
As read on 9 Oct 2026, 1,987 bytes. 16 rule groups name AI crawlers; shown first.
User-agent: GPTBot Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: anthropic-ai Disallow: / User-agent: Claude-Web Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Google-Extended Disallow: / User-agent: CCBot Disallow: / User-agent: PerplexityBot Disallow: / User-agent: FacebookBot Disallow: / User-agent: Omgili Disallow: / User-agent: Omgilibot Disallow: / User-agent: Bytespider Disallow: / User-agent: cohere-ai Disallow: / User-agent: Applebot-Extended Disallow: / User-agent: Amazonbot Disallow: / User-agent: Diffbot Disallow: / User-agent: * Disallow: /forgotpw/* User-agent: * Disallow: /auth/user/login/* User-agent: * Disallow: /myprofile/* User-agent: * Disallow: /subscription/* User-agent: * Disallow: /test_frontpage User-agent: * Disallow: /front User-agent: * Disallow: /annonsorinnhold* User-agent: * Disallow: /sponsor-content* User-agent: * Disallow: /advertise/* User-agent: facebookexternalhit Allow: /annonsorinnhold* User-agent: Facebot Allow: /annonsorinnhold* User-agent: LinkedInBot Allow: /annonsorinnhold* User-agent: Twitterbot Allow: /annonsorinnhold* User-agent: facebookexternalhit Allow: /sponsor-content* User-agent: Facebot Allow: /sponsor-content* User-agent: LinkedInBot Allow: /sponsor-content* User-agent: Twitterbot Allow: /sponsor-content* User-agent: UbiCrawler Disallow: / User-agent: DOC Disallow: / User-agent: Zao Disallow: / User-agent: Zealbot Disallow: / User-agent: MSIECrawler Disallow: / User-agent: SiteSnagger Disallow: / User-agent: WebStripper Disallow: / User-agent: WebCopier Disallow: / User-agent: Fetch Disallow: / User-agent: Offline Explorer Disallow: / User-agent: Teleport Disallow: / User-agent: TeleportPro Disallow: / User-agent: linko Disallow: / User-agent: Microsoft.URL.Control Disallow: / User-agent: wget Disallow: / User-agent: grub-client Disallow: / User-agent: WebReaper Disallow: / Sitemap: https://www.tradewindsnews.com/sitemap.xml
Changes recorded since June 2026.
No change recorded.