CrawlerPulse
CrawlerPulse / Europe / Latvia / novayagazeta.eu

Novaya Gazeta Europe

novayagazeta.eu · Latvia · measured 9 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 9 Oct 2026, 1,291 bytes. No rule names an AI crawler.

User-agent: *
  Disallow: /api/
  Disallow: /hyper/
  Disallow: /preview/
  Disallow: /cdn-cgi/
  User-agent: Twitterbot
  Disallow:
  User-agent: Yandex
  Disallow: /api/
  Disallow: /hyper/
  Disallow: /preview/
  Disallow: /cdn-cgi/
  Clean-param: utm_source *
  Clean-param: from *
  Clean-param: fromtg *
  Sitemap: https://novayagazeta.eu/feed/sitemap/pages.xml
Sitemap: https://novayagazeta.eu/feed/sitemap/authors.xml
Sitemap: https://novayagazeta.eu/feed/sitemap/topics.xml
Sitemap: https://novayagazeta.eu/feed/sitemap/categories.xml
Sitemap: https://novayagazeta.eu/feed/sitemap/articles-daily.xml
Sitemap: https://novayagazeta.eu/feed/sitemap/articles-weekly.xml
Sitemap: https://novayagazeta.eu/feed/sitemap/articles-1.xml
Sitemap: https://novayagazeta.eu/feed/sitemap/articles-2.xml
Sitemap: https://novayagazeta.eu/feed/sitemap/articles-3.xml
Sitemap: https://novayagazeta.eu/feed/sitemap/articles-4.xml
Sitemap: https://novayagazeta.eu/feed/sitemap/articles-5.xml
Sitemap: https://novayagazeta.eu/feed/sitemap/articles-6.xml
Sitemap: https://novayagazeta.eu/feed/sitemap/articles-7.xml
Sitemap: https://novayagazeta.eu/feed/sitemap/articles-8.xml
Sitemap: https://novayagazeta.eu/feed/sitemap/articles-9.xml
Sitemap: https://novayagazeta.eu/feed/sitemap/articles-10.xml

History

Changes recorded since June 2026.

No change recorded.

In context

Latvia
19.2% block at least one AI crawler
Behind Cloudflare
Yes