CrawlerPulse
CrawlerPulse / Europe / Russia / sportbox.ru

RTR

sportbox.ru · Russia · RTR - Russia (Russian) · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 1,560 bytes. No rule names an AI crawler.

User-agent: *
Allow: /
Disallow: /includes
Disallow: /misc
Disallow: /modules
Disallow: /profiles
Disallow: /scripts
Disallow: /themes
Disallow: /short_game
Disallow: /spb_conf
Disallow: /ipbcomments
Disallow: /spb_stats/match
Disallow: /vdl
Disallow: /Text
Disallow: /node
Disallow: /profile
Disallow: /people
Disallow: /viewstats
Disallow: /sites/default/files/
Disallow: /taxonomy
Disallow: /konkurs/globalrating
Disallow: /stats/konkurs/*
Disallow: /fotobank
Disallow: /stats/game_json/*
Disallow: /ums_signon
Disallow: /ums_registration
Disallow: /ums_password_restore
Disallow: /*/stats/*page_offset=*&page_size=
Disallow: /second-screenVidy_sporta
Disallow: /pm/login/
Disallow: /gid/login/
Disallow: /*/stats*/konkurs/stage_
Disallow: *amppage_offset
Disallow: *amptournament_id
Disallow: *amppage_size
Clean-param: time&channel /video
Clean-param: page&uid&p&nid&only_player&d&playeri&ref&tour /
Clean-param: nickname&do&x_real_path&ipbdescription&rnd&popup_comments /
Clean-param: web_optimizer_debug&imgDelta&id&comments_number&huge&pitem&cid /
Clean-param: nid&gres&usg&ved&sa&ei&results&place&start&type&doc_id /
Clean-param: tour&field_object_published_start&section&w&keys /
Clean-param: field_issue_date&date&field_tag&archive&mobile_comments /
Clean-param: sp&tyear&turnir&field_channel_id /
Clean-param: FAn22&ncrnd&field_channel_id-empty-marker /
Clean-param: utm_source&utm_medium&time&channel&autostart /
Clean-param: adfox&from&clid /
Clean-param: alex&jhk&re_&re&___&ol&Alex&-&%2F%3F&r&pp /
Clean-param: trid&ts&MSISDN&_sm_byp&msclkid /

History

Changes recorded since June 2026.

No change recorded.

In context

Russia
6.3% block at least one AI crawler
Behind Cloudflare
No