CrawlerPulse
CrawlerPulse / Europe / Russia / vl.ru

VL.ru

vl.ru · Russia · measured 9 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 9 Oct 2026, 2,651 bytes. No rule names an AI crawler.

User-agent: AhrefsBot
Disallow: /

User-agent: *
Disallow: /ajax/
Disallow: /admin/
Disallow: /commentsgate/
Allow: /commentsgate/ajax/thread/
Disallow: /extauth/
Disallow: /forums/search.php
Disallow: /forums/login.php
Disallow: /forums/post.php
Disallow: /forums/profile.php
Disallow: /help$
Disallow: /help/
Disallow: *letter=*
Disallow: /login*
Disallow: /main/
Disallow: /moi-vlki/
Disallow: /afisha/admin
Disallow: /afisha/purchase/
Disallow: /afisha/khabarovsk/
Disallow: /afisha/*?page=
Disallow: /afisha/*/search?query=*
Disallow: /bilet/
Disallow: /booking-grid/*
Disallow: /dengi/credits/calculator/?
Disallow: /dengi/investments/savings/calculator?
Disallow: /detki/exchange/api/
Disallow: /detki/api/
Disallow: /detki/photoreport/*/comment-add
Disallow: /detki/*?lib=
Disallow: /eda/owner
Disallow: /eda/admin
Disallow: /eda/subscribe
Disallow: /eda/search
Disallow: /eda/get-app
Disallow: /med/*?_page=*
Disallow: /med/*?page=*
Disallow: /newyear-favorites/
Disallow: /off/map/
Disallow: /off/pd.html
Disallow: /off/statement/form
Disallow: /off/statement/form.html
Disallow: /pusher
Disallow: /registration/
Disallow: /sections/?page=*
Allow: /transport/search?*&day=&ts=0&te=24*
Allow: /transport/search?*&day=$
Disallow: /transport/search?*&day=*
Disallow: /transport/search?*&ts=*
Disallow: /transport/search?*&te=*
Disallow: /transport/gas/*/*/*
Disallow: /transport/*route/
Disallow: /transport/tickets/bus/purchase/*
Disallow: /transport/tickets/bus/*/*?date=*
Disallow: /transport/transfer?*subroutes*
Disallow: /tv/feedback
Disallow: /tv/grid*time=*
Disallow: /tv/events*time=*
Disallow: /tv/channel.php*
Disallow: /tv/timesort.php*
Disallow: /tv/sitemap-8.txt
Disallow: /tv/sitemap-6.txt
Disallow: /tv/sitemap-5.txt
Disallow: /tv/sitemap-7.txt
Disallow: /vote$
Disallow: /vote/
Disallow: /whois
Disallow: /afisha/pdf/
Disallow: /eda/api/
Disallow: /eda/ajax/
Disallow: /eda/external/
Disallow: /weather/api/
Disallow: /favorites
Disallow: /travel?*utm_id=
Disallow: /*?dsl=r
Disallow: /vladivostok?search=
Disallow: /polling/archive/main-vl/
Disallow: /*?from=novelty
Disallow: /*?buildingId=
Disallow: /*/blackout-history/content?filter=
Disallow: /off/interview/
Disallow: /off/interviews
Disallow: /off/bundles/
Disallow: /off/build/
Disallow: /off/blackout/
Disallow: /afisha/*/events/by-dates-
Disallow: /afisha/*/personal/
Clean-param: utm_source&utm_campaign&utm_term&utm_medium&utm_content&_ga&author
Clean-param: route_number
Clean-param: day
Host: https://www.vl.ru
Sitemap: https://www.vl.ru/afisha/sitemap/sitemap_vl_index.xml
Sitemap: https://www.vl.ru/off/sitemap-2.txt
Sitemap: https://www.vl.ru/tv/sitemap.txt

History

Changes recorded since June 2026.

No change recorded.

In context

Russia
6.3% block at least one AI crawler
Behind Cloudflare
No