lrytas.lt · Lithuania · lrytas.lt / Naujienos · measured 8 Oct 2026
As read on 8 Oct 2026, 1,915 bytes. 1 rule group name AI crawlers; shown first.
User-agent: Mozilla/5.0 (compatible;PetalBot;+https://webmaster.petalsearch.com/site/petalbot) Disallow: / User-agent: * Disallow: */email.asp Disallow: */print.asp Disallow: *view=* User-agent: CazoodleBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: dotbot/1.0 Disallow: / User-agent: Gigabot Disallow: / User-agent: trendkite-akashic-crawler Disallow: / User-agent: AhrefsBot Disallow: / User-agent: ias-va Disallow: / User-agent: ias-va/3.1 Disallow: / User-agent: ias- Disallow: / User-agent: BLEXBot/1.0 Disallow: / User-agent: ias-jp/3.1 Disallow: / User-agent: ias-sg/3.1 Disallow: / User-agent: trendkite-akashic-crawler Disallow: / User-agent: Mozilla/5.0 (compatible; GrapeshotCrawler/2.0; +http://www.grapeshot.co.uk/crawler.php) Disallow: / User-agent: CriteoBot/0.1 (+https://www.criteo.com/criteo-crawler/) Disallow: / User-agent: Mozilla/5.0 (compatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com) Disallow: / User-agent: Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/2.0; +http://go.mail.ru/help/robots) Disallow: / User-agent: Mozilla/5.0 (compatible; DataForSeoBot/1.0; +https://dataforseo.com/dataforseo-bot) Disallow: / User-agent: Mozilla/5.0 (compatible; archive.org_bot +http://archive.org/details/archive.org_bot) Disallow: / User-agent: Cutbot; 1.5; http://cutbot.net/ Disallow: / User-agent: AhrefsBot Disallow: / User-agent: Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/) Disallow: / User-agent: Mozilla/5.0 (compatible; BLEXBot/1.0; +http://webmeup-crawler.com/) Disallow: / User-agent: YandexBot/3.0 Disallow: / User-agent: * Disallow: */comments*
Changes recorded since June 2026.
No change recorded.