mexicoledger.com · United States · USAToday · measured 8 Oct 2026
As read on 8 Oct 2026, 1,938 bytes. 1 rule group name AI crawlers; shown first.
User-Agent: omgilibot Disallow: / User-agent: Mediapartners-Google Disallow: User-agent: Magnetbot Disallow: User-agent: Yandex Crawl-delay: 3 User-agent: Baiduspider Crawl-delay: 3 User-agent: bingbot Crawl-delay: 1 User-agent: Slurp Crawl-delay: 2 User-Agent: Genieo Disallow: / User-Agent: ecoResearch Disallow: / User-Agent: Spinn3r Disallow: / User-Agent: BLP_bbot Disallow: / User-Agent: R6_CommentReader Disallow: / User-Agent: EasouSpider Disallow: / User-Agent: A6-Indexer Disallow: / User-Agent: magpie-crawler Disallow: / User-agent: * Disallow: /css/ Disallow: /css_system/ Disallow: /js/ Disallow: /js_system/ Disallow: /ajax/ Disallow: /searchmode Disallow: /taxonomy Disallow: /marketplace Disallow: /classifieds Disallow: /premium/ Disallow: /premium/agriculture-news Disallow: /premium/automotive-news Disallow: /premium/books-news Disallow: /premium/business-news Disallow: /premium/education-careers Disallow: /premium/entertainment-news Disallow: /premium/food-news Disallow: /premium/garden-news Disallow: /premium/green-living Disallow: /premium/home-news Disallow: /premium/kids-family Disallow: /premium/lifestyle-news Disallow: /premium/money-matters Disallow: /premium/outdoors-news Disallow: /premium/pets-news Disallow: /premium/puzzles Disallow: /premium/real-estate-news Disallow: /premium/seniors-news Disallow: /premium/spanish-news Disallow: /premium/tech-news Disallow: /premium/travel-news Disallow: /premium/games-news Disallow: /premium/health-news Disallow: /calendar
Changes recorded since June 2026.
No change recorded.