worldsoccertalk.com · United States · measured 9 Oct 2026
As read on 9 Oct 2026, 1,780 bytes. No rule names an AI crawler.
User-agent: * Allow: / Sitemap: https://worldsoccertalk.com/sitemaps/news/ Sitemap: https://worldsoccertalk.com/sitemaps/images/ Sitemap: https://worldsoccertalk.com/sitemaps/author-sitemap/ Sitemap: https://worldsoccertalk.com/sitemaps/xml/evergreen/category/pages/ User-agent: MSIECrawler Disallow: / User-agent: WebCopier Disallow: / User-agent: HTTrack Disallow: / User-agent: Microsoft.URL.Control Disallow: / User-agent: libwww Disallow: / User-agent: Orthogaffe Disallow: / User-agent: UbiCrawler Disallow: / User-agent: DOC Disallow: / User-agent: Zao Disallow: / User-agent: sitecheck.internetseer.com Disallow: / User-agent: Zealbot Disallow: / User-agent: MSIECrawler Disallow: / User-agent: SiteSnagger Disallow: / User-agent: WebStripper Disallow: / User-agent: WebCopier Disallow: / User-agent: Fetch Disallow: / User-agent: Offline Explorer Disallow: / User-agent: Teleport Disallow: / User-agent: TeleportPro Disallow: / User-agent: WebZIP Disallow: / User-agent: linko Disallow: / User-agent: HTTrack Disallow: / User-agent: Microsoft.URL.Control Disallow: / User-agent: Xenu Disallow: / User-agent: larbin Disallow: / User-agent: libwww Disallow: / User-agent: ZyBORG Disallow: / User-agent: Download Ninja Disallow: / User-agent: wget Disallow: / User-agent: grub-client Disallow: / User-agent: k2spider Disallow: / User-agent: NPBot Disallow: / User-agent: WebReaper Disallow: / User-agent: Googlebot Disallow: /trends/ Disallow: /api/request-image/ User-agent: Googlebot-News Disallow: /sponsored/ User-agent: * Disallow: /contenidos/ User-agent: Bingbot Disallow: /api/request-image/
Changes recorded since June 2026.
No change recorded.