castanet.net · Canada · Glacier Media · measured 8 Oct 2026
As read on 8 Oct 2026, 2,332 bytes. 3 rule groups name AI crawlers; shown first.
User-agent: Bytespider Disallow: / User-agent: GPTBot Disallow: / User-agent: omgili Disallow: / User-agent: Mediapartners-Google Disallow: User-agent: AhrefsBot Disallow: / User-agent: AwarioBot Disallow: / User-agent: CAAM Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: DAUM Disallow: / User-agent: VocusBot Disallow: / User-agent: FlipboardProxy Disallow: /cgi-bin2/ User-agent: G-i-g-a-b-o-t Disallow: / User-agent: GoodBot Disallow: / User-agent: grapeshot Disallow: / User-agent: Linguee Bot Disallow: / User-agent: LinkpadBot Disallow: / User-agent: LinkupBot Disallow: / User-agent: MauiBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: weborama-fetcher Disallow: / User-agent: Yandex Disallow: / User-agent: Yandexbot Disallow: / User-agent: Yeti Disallow: / User-agent: BLEXBot Disallow: / User-agent: Seekport Crawler Disallow: / User-agent: SeekportBot Disallow: / User-agent: SentiBot Disallow: / User-agent: SEOkicks Disallow: / User-agent: trendkite-akashic-crawler Disallow: / User-agent: archive.org_bot Disallow: /cgi-bin2/ User-agent: rogerbot Disallow: /cgi-bin2/ User-agent: search.marginalia.nu Disallow: /cgi-bin2/ User-agent: Claude-SearchBot Disallow: /contactus/report_typo.php User-agent: * Disallow: /admin/ Disallow: /sales/ Disallow: /postcomment.php Disallow: /viewcomments.php Disallow: /cgi-bin/ Allow: /templates/weather/images/ Disallow: /templates/ Disallow: /ccs/ Disallow: /lists/ Disallow: /logs/ Disallow: /menus/ Disallow: /quote/ Disallow: /survey/ Disallow: /traffic/ Disallow: /webmail/ Disallow: /events_dev/ Disallow: /demo.htm Disallow: /search.htm Disallow: /crash.htm Disallow: /index-jc.htm Disallow: /chances.htm Disallow: /chances1.htm Disallow: /boats.htm Disallow: /v.htm Disallow: /index-irish.htm Disallow: /index-noirish.htm Disallow: /index-troy.htm Disallow: /index-troy2.htm Disallow: /troy/ Disallow: /homepage/ Disallow: /fred/ Disallow: /real-v2/ Disallow: /refresh/ Disallow: /widgets/ Disallow: /wine/example/ Disallow: /realestatenew/ Disallow: /cat/marketplace/home_leisure/store_-_hot_tub_essentials/ Disallow: /clients/img/ Disallow: /realestate-old/ Disallow: /ryan/ Disallow: /h/ Disallow: /r/ Disallow: /1/ Disallow: /feeds/ Disallow: /cdn-cgi/ Disallow: /TCadmin/ Disallow: /garagesalesnew/ Disallow: /adpreview/ Disallow: /wineries/
Changes recorded since June 2026.
No change recorded.