CrawlerPulse
CrawlerPulse / Europe / Russia / myslo.ru

Sloboda

myslo.ru · Russia · Sloboda - Russia · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 815 bytes. No rule names an AI crawler.

User-Agent: *
Disallow: *?*
Disallow: /admin
Disallow: /home
Disallow: /redirect
Disallow: /search
Disallow: /user
Disallow: /Admin
Disallow: /Home
Disallow: /Redirect
Disallow: /Search
Disallow: /User
Disallow: /NewsAdditionalColumn
Disallow: /comments/goto/*
Disallow: /Content/Banners/*
Disallow: *comment_*
Disallow: /pages/gorodskie-sumashedshie
Disallow: /news/sport/stadion-fk-arsenal-pereydet-v-sobstvennost-goroda
Disallow: /news/tula/obeshchalki-2015
Disallow: /news/tula/v-tule-budut-stroit-noviy-stadion
Disallow: /Content/Pressa/Pdf/56/c6/56c64cd8-0a46-4684-a966-85c2f3427b6a.pdf
Disallow: /Content/Pressa/Pdf/49/2b/492ba775-548e-4ac4-889f-d92c849db296.pdf
Host: https://myslo.ru
Sitemap: https://myslo.ru/sitemap.xml

User-agent: YandexNews
Allow: /rss/newsyandex

History

Changes recorded since June 2026.

No change recorded.

In context

Russia
6.3% block at least one AI crawler
Behind Cloudflare
No