CrawlerPulse
CrawlerPulse / Europe / Russia / primamedia.ru

PrimaMedia

primamedia.ru · Russia · measured 9 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 9 Oct 2026, 1,231 bytes. No rule names an AI crawler.

User-agent: *
Allow: /
Disallow: /aboutus
Disallow: /p3
Disallow: /p3_08.04
Disallow: /p3_20.05
Disallow: /admin
Disallow: /travel/old
Disallow: /forum
Disallow: /news/send/
Disallow: /news/save/
Disallow: /news/translate/
Disallow: /news.htm
Disallow: /news.txt
Disallow: /vladivostoktimes
Disallow: /afisha
Disallow: /economics
Disallow: /sport
Disallow: /apec
Disallow: /push
Disallow: /connect
Disallow: /connect/cp
Disallow: /pda
Disallow: /zakup
Disallow: /wap
Disallow: /old
Disallow: /test
Disallow: /phpajax
Disallow: /_oldonline
Disallow: /_online
Disallow: /search
Disallow: /news/eao
Disallow: /news/birobidjan
Disallow: /news/khabarovsk
Disallow: /news/khabkrai
Disallow: /news/irkutsk
Disallow: /news/oblast
Disallow: /news/sakhalin/
Disallow: /news/island/
Disallow: /pda
Disallow: /id
Disallow: /id/*
Disallow: /auto/badcopslist.php
Disallow: /*/?nocache
Disallow: /news/*preview=*
Crawl-delay: 100
Host: https://primamedia.ru
Sitemap: https://primamedia.ru/sitemap/
Clean-param: utm_source&utm_medium&utm_campaign&from&to&amp_lite&__twitter_impression&f_firm&y&lang

User-agent: Yandex
Crawl-delay: 30

User-agent: MSNbot
Crawl-Delay: 120

User-agent: Slurp
Crawl-Delay: 100

User-agent: googlebot
crawl-delay: 30

History

Changes recorded since June 2026.

No change recorded.

In context

Russia
6.3% block at least one AI crawler
Behind Cloudflare
No