CrawlerPulse
CrawlerPulse / Europe / Russia / rbc.ru

RBC Information Systems

rbc.ru · Russia · measured 9 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 9 Oct 2026, 4,696 bytes. No rule names an AI crawler.

User-Agent: *
Disallow: /*?q=
Disallow: /search/
Disallow: /search?query=
Disallow: /search?q=
Disallow: /info-service/api/v2/ev/
Disallow: /cgi-bin/
Disallow: /search2/
Disallow: /include/
Disallow: /includes/
Disallow: *page*
Disallow: /photoslider/
Disallow: /commercial-preview/
Disallow: /ajax/
Disallow: */search
Disallow: */preview/
Disallow: /companies/
Disallow: /quote/user/
Disallow: /quote/login/
Disallow: /quote/registration/
Disallow: /quote/forecast/
Disallow: /edu/
Disallow: /courses/goto?type=*
Disallow: /courses/*?sort_by=*
Disallow: /courses/*?schools*
Disallow: /courses/*?durations*
Disallow: /courses/*?has_installment*
Disallow: /courses/*?high_rating*
Disallow: /courses/*?license*
Disallow: /courses/login
Disallow: /courses/register

User-agent: GoogleBot
Disallow: /*?q=
Disallow: /search/
Disallow: /search?query=
Disallow: /search?q=
Disallow: /info-service/api/v2/ev/
Disallow: /cgi-bin/
Disallow: /search2/
Disallow: /include/
Disallow: /includes/
Disallow: *page*
Disallow: /commercial-preview/
Disallow: /ajax/
Disallow: */search
Disallow: */preview/
Disallow: /companies/
Disallow: /quote/user/
Disallow: /quote/login/
Disallow: /quote/registration/
Disallow: /quote/forecast/
Disallow: /edu/
Disallow: /courses/goto?type=*
Disallow: /courses/*?sort_by=*
Disallow: /courses/*?schools*
Disallow: /courses/*?durations*
Disallow: /courses/*?has_installment*
Disallow: /courses/*?high_rating*
Disallow: /courses/*?license*
Disallow: /courses/login
Disallow: /courses/register

User-agent: Yandex
Disallow: /quote/user/
Disallow: /quote/login/
Disallow: /quote/registration/
Disallow: /*?q=
Disallow: /search/
Disallow: /search?query=
Disallow: /search?q=
Disallow: /info-service/api/v2/ev/
Disallow: /cgi-bin/
Disallow: /search2/
Disallow: /include/
Disallow: /includes/
Disallow: *page*
Disallow: /photoslider/
Disallow: /commercial-preview/
Disallow: /ajax/
Disallow: */search
Disallow: */preview/
Disallow: /companies/
Disallow: /quote/forecast/
Disallow: */amp/
Disallow: /edu/
Disallow: /courses/goto?type=*
Disallow: /courses/*?sort_by=*
Disallow: /courses/*?schools*
Disallow: /courses/*?durations*
Disallow: /courses/*?has_installment*
Disallow: /courses/*?high_rating*
Disallow: /courses/*?license*
Disallow: /courses/login
Disallow: /courses/register
Clean-param: from *
Clean-param: utm_source *
Clean-param: google_editors_picks *
Clean-param: xtor *
Clean-param: print *
Clean-param: google_preview *
Clean-param: nw *
Clean-param: fromtg *
Clean-param: utm_referrer&source&erid&utm_partner_id&_sm_byp&fbclid&foxConfig.debug&video_id&msclkid&ref&mibextid&ext&target&t&HostingCenter&hl&pm-test-form&MSISDN&keepThis&CAT&mbstx&TB_iframe&aim-banner-id&s
Clean-param: RULE&DATETIME&FILE&CODE&CLIENTID&LANG&optcheckwfsp&URL&IP&ui&USER&ga_client_id&back&as_qdr&as_occt&safe&as_q&channel&width&height&frm&tid&_cldee&recipientid&mindbox-message-key&utm_medium&utm_campaign
Clean-param: noredir&utm_p&mc_cid&mc_eid&utm_test&click_id&clid&utm_site&email&rbc_type&spush&utm_reqid&story&grs&espv&caption&_ym_debug&_hsmi=&_hsenc&_hsmi&sa&_sm_au_&iu&usqp&ved&gdfp_req&lineItemId&creativeId&nsukey&platformId&ysclid
Clean-param: amp&okcppax&daKillaOfZeeCache&text&parent-reqid&__rm_debug&mkt_tok&gcmes&lite&uid&fireglass_rsn&a&disableGlobalInfoCollect&event_name&id&vpaid&rolltype&cbb&suggest_reqid&dm_i&trk&keepThis&TB_iframe
Clean-param: pulse_traffic&tg_rhash&post&no_header&ocid&mgclid&__FB_PRIVATE_TRACKING__&yptr&linkId&bx_sender_conversion_id&utm_med&GarantPark&ampcf&muid&retpath&smclient&home_board&mobile_style&etm_term
Clean-param: birth_date&is_male&region_id&positions&_sc_token&category&vk&fs&el&hcb&_gl&sfns&check_swipe&v&o&r&c&xrs&gcmlg&zarsrc&refresh&isappinstalled&ct&gidzl&variant&goal&_gclid&height&width&caption
Clean-param: bulk_email_rid&bpmtrackid&bpmreplica&contactId&tpclid&stat-id&test-tag&banner-sizes&actual-format&pcodever&adsdk-bundle-version&banner-test-tags&constructor-rendered-assets&ldc&rs&oe&op&dt&fu&if
Clean-param: continueFlag&_x_tr_sl&_x_tr_tl&_x_tr_hl&_x_tr_pto&param&cmd&arubalp&pubsrcid&pubsrcid&push_id&utm_sup_source&utm_sup_project&utm_sup_type&ssp&setlang&safesearch&rb_clickid&realUrl&nsukey&user&yptr
Clean-param: win&yredirect&articleId&user&media_source&menu&usg&ust&bp_origin&inu&darkschemeovr&openTree&utf&preview&etext&shem&_cio_id&roistat_visit&tip&fid&FacebookPixel&wmode&wv-part&wv-type&browser-info&reference&status
Clean-param: bulkEmailRecipientId&ruid&__fd__&w&partner&nid&spm&slide&homescreen&closeAction
Clean-param: s&referrer&url_next&tab
Sitemap: https://www.rbc.ru/sitemap_index.xml

History

Changes recorded since June 2026.

No change recorded.

In context

Russia
6.3% block at least one AI crawler
Behind Cloudflare
No