CrawlerPulse
CrawlerPulse / Europe / Russia / eg.ru

Express Gazeta

eg.ru · Russia · measured 9 Oct 2026

Google Search
Partial
Google-Extended
Partial
OpenAI
Partial
Anthropic
Partial
Perplexity
Partial
Meta
Partial
Common Crawl
Partial
ByteDance
Partial
Apple
Partial
Amazon
Partial

robots.txt

As read on 9 Oct 2026, 1,391 bytes. No rule names an AI crawler.

User-agent: Yandex
Disallow: /gsearch/
Disallow: /article/*
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content/plugins
Disallow: /wp-login.php
Disallow: /cgi-bin
Disallow: /*.pdf$
Disallow: /*author-id=*
Disallow: /*photo_id=*
Disallow: */amp/
Disallow: /author/*
Disallow: /daily/*
Disallow: /sitemap-relax.xml
Disallow: /sitemap-showbusiness23.xml
Disallow: /sitemap-showbusiness89.xml
Disallow: /sitemap-showbusiness45.xml
Disallow: /sitemap-showbusiness01.xml
Disallow: /sitemap-showbusiness67.xml
Disallow: /sitemap-society56789.xml
Disallow: /sitemap-nostalgia.xml
Disallow: /sitemap-society01234.xml
Disallow: /sitemap-politics.xml
Disallow: /sitemap-other.xml
Disallow: /sitemap-*

User-agent: Googlebot
Disallow: /gsearch/
Disallow: /article/*
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content/plugins
Disallow: /wp-login.php
Disallow: /cgi-bin
Disallow: /*.pdf$
Disallow: /*author-id=*
Disallow: /*photo_id=*
Disallow: /*utm=short
Disallow: /daily/*
Disallow: /sitemap-*

User-agent: *
Disallow: /*?
Disallow: /gsearch/
Disallow: /article/*
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content/plugins
Disallow: /wp-login.php
Disallow: /cgi-bin
Disallow: /*.pdf$
Disallow: /*author-id=*
Disallow: /*photo_id=*
Disallow: /daily/*
Clean-param: pubsrcid /
Clean-param: _utl_t /
Clean-param: utm /
Clean-param: utm_p /
Host: https://www.eg.ru

History

Changes recorded since June 2026.

No change recorded.

In context

Russia
6.3% block at least one AI crawler
Behind Cloudflare
No