CrawlerPulse
CrawlerPulse / Europe / Poland / tokfm.pl

TOKFM

tokfm.pl · Poland · Agora PL · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Blocked
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Blocked
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 1,402 bytes. 3 rule groups name AI crawlers; shown first.

User-agent: GPTBot
Disallow: /

User-agent: CCBot
Disallow: /

User-agent: anthropic-ai
Disallow: /

User-agent: grapeshot
Disallow:

User-agent: *
Disallow: /settings/
Disallow: /user/
Disallow: /Zastrzezenia-Prawne
Disallow: /Wyszukaj
Disallow: /Multimedia/
Disallow: /esi/block/*
Disallow: /Elementy-strony/*
Disallow: /Elementy-strony2/
Disallow: /layout/*
Disallow: /var/embeds/instagram/*.json
Disallow: */(quantcastcard)/*
Disallow: /go/*
Disallow: /quiz/getJson/
Disallow: */?_gl=*
Disallow: /media/galeria/*
Disallow: /media/quiz/*
Disallow: */utm_source=
Disallow: */(galeria_element)/*
Disallow: */(quiz_element)/*
Disallow: /content/view/full/*
Disallow: /?s=*
Disallow: /?tag=*
Allow: /esi/block/seo/sitemap*

History

Changes recorded since June 2026.

No change recorded.

In context

Poland
36.4% block at least one AI crawler
Behind Cloudflare
No
Same group