CrawlerPulse
CrawlerPulse / Europe / Poland / wyborcza.pl

Wyborcza

wyborcza.pl · Poland · Agora PL · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Blocked
Anthropic
Blocked
Perplexity
Blocked
Meta
Allowed
Common Crawl
Blocked
ByteDance
Blocked
Apple
Blocked
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 14,241 bytes. 5 rule groups name AI crawlers; shown first.

User-agent: ChatGPT-User
Disallow: /

User-agent: GPTBot
Disallow: /

User-agent: Mozilla/5.0 (compatible; ImagesiftBot; +imagesift.com)
Disallow: /

User-agent: PetalBot
Disallow: /

User-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5 (Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot),"156,975"
Disallow: /

User-agent: *
Disallow: /ot-amp-consent
Disallow: /*/wyszukaj/
Disallow: /*servlet
Disallow: /reloadwww?
Disallow: /dfptools/adview/
Disallow: /pub/ips/*
Disallow: /ods?
Disallow: /getFile.servlet*
Disallow: /aliasy/blad.jsp
Disallow: /znajdz.do
Disallow: /portalSearch.do
Disallow: /im/ab/b4/10/z17515435Q.jpg
Disallow: /75224259/
Disallow: /opinions20/
Disallow: /duzyformat/1,130280,13165330,REGULAMIN_UDZIALU_W_KONKURSIE___JOLKA_W_DUZYM_FORMACIE__.html
Disallow: /1,75478,17068768,CBA_zatrzymalo_wysokich_urzednikow_ZUS__Chodzi_o_przetargi.html
Disallow: /bialystok/1,35241,19674535*
Disallow: /bialystok/1,35241,19697480*
Disallow: /paypath/
Disallow: /lpages/
Disallow: /pages/tweaks/stattrack_article.html
disallow: /*/multimedia/hr360/med/index.html?pano=data:text*

User-agent: Googlebot-News
Disallow: /nowy/
Disallow: /mapa_strony
Disallow: /*/wyszukaj/
Disallow: /*/51,
Disallow: /*/55,
Disallow: /*/2,
Disallow: /*order=
Disallow: /*obxx=
Disallow: /*tag=
Disallow: /reloadwww?
Disallow: /ods?
Disallow: /*servlet
Disallow: /dfptools/adview/
Disallow: /opinions20/

User-agent: 360Spider
Disallow: /

User-agent: 80legs
Disallow: /

User-agent: Aboundex
Disallow: /

User-agent: accelobot
Disallow: /

User-agent: Add\ Catalog
Disallow: /

User-agent: aiHitBot
Disallow: /

User-agent: Alexibot
Disallow: /

User-agent: Aqua_Products
Disallow: /

User-agent: AskJeeves
Disallow: /

User-agent: asterias
Disallow: /

User-agent: awcheckBot
Disallow: /

User-agent: b2w/0.1
Disallow: /

User-agent: BackDoorBot/1.0
Disallow: /

User-agent: BacklinkCrawler
Disallow: /

User-agent: Baiduspider
Disallow: /

User-agent: Barkrowler
Disallow: /

User-agent: BecomeBot
Disallow: /

User-agent: BLEXBot
Disallow: /

User-agent: BlowFish/1.0
Disallow: /

User-agent: Bookmark search tool
Disallow: /

User-agent: BotALot
Disallow: /

User-agent: brandwatch.net
Disallow: /

User-agent: BuiltBotTough
Disallow: /

User-agent: Bullseye/1.0
Disallow: /

User-agent: BunnySlippers
Disallow: /

User-agent: Butterfly
Disallow: /

User-agent: CatchBot
Disallow: /

User-agent: Charlotte
Disallow: /

User-agent: CheeseBot
Disallow: /

User-agent: CherryPicker
Disallow: /

User-agent: CherryPickerElite/1.0
Disallow: /

User-agent: CherryPickerSE/1.0
Disallow: /

User-agent: Clickagy Intelligence Bot v2
Disallow: /

User-agent: CLIPish
Disallow: /

User-agent: Cliqzbot
Disallow: /

User-agent: COMODO
Disallow: /

User-agent: Comodo-Certificates-Spider
Disallow: /

User-agent: CompSpyBot
Disallow: /

User-agent: Copernic
Disallow: /

User-agent: CopyRightCheck
Disallow: /

User-agent: cosmos
Disallow: /

User-agent: crawler
Disallow: /

User-agent: Crescent
Disallow: /

User-agent: Crescent Internet ToolPak HTTP OLE Control v.1.0
Disallow: /

User-agent: Curious
Disallow: /

User-agent: curl
Disallow: /

User-agent: DataForSeoBot
Disallow: /

User-agent: dataprovider\.com
Disallow: /

User-agent: DinoPing
Disallow: /

User-agent: discoverybot
Disallow: /

User-agent: DittoSpyder
Disallow: /

User-agent: DomainCrawler
Disallow: /

User-agent: dotbot
Disallow: /

User-agent: dotnetdotcom
Disallow: /

User-agent: Dow\ Jones\ Searchbot
Disallow: /

User-agent: dumbot
Disallow: /

User-agent: EasouSpider
Disallow: /

User-agent: EmailCollector
Disallow: /

User-agent: EmailSiphon
Disallow: /

User-agent: EmailWolf
Disallow: /

User-agent: Enterprise_Search
Disallow: /

User-agent: Enterprise_Search/1.0
Disallow: /

User-agent: EroCrawler
Disallow: /

User-agent: es
Disallow: /

User-agent: Exabot
Disallow: /

User-agent: ExtractorPro
Disallow: /

User-agent: EyeMonIT
Disallow: /

User-agent: EzineArticlesLinkScanner
Disallow: /

User-agent: Ezooms
Disallow: /

User-agent: FairAd Client
Disallow: /

User-agent: Feedpresso Content Index Bot
Disallow: /

User-agent: Flaming AttackBot
Disallow: /

User-agent: Foobot
Disallow: /

User-agent: FreeFind
Disallow: /

User-agent: FTRF\:\ Friendly
Disallow: /

User-agent: Gaisbot
Disallow: /

User-agent: GetRight/4.2
Disallow: /

User-agent: gigabot
Disallow: /

User-agent: grub
Disallow: /

User-agent: grub-client
Disallow: /

User-agent: Harvest/1.5
Disallow: /

User-agent: Hatena Antenna
Disallow: /

User-agent: hloader
Disallow: /

User-agent: http://linkfluence.com
Disallow: /

User-agent: http://mediacloud.org
Disallow: /

User-agent: http://www.SearchEngineWorld.com  bot
Disallow: /

User-agent: http://www.WebmasterWorld.com  bot
Disallow: /

User-agent: HTTP_Request
Disallow: /

User-agent: HTTP_Request2
Disallow: /

User-agent: httplib
Disallow: /

User-agent: https://developers.snap.com/robots
Disallow: /

User-agent: https://help.qwant.com/bot/
Disallow: /

User-agent: humanlinks
Disallow: /

User-agent: ia_archiver
Disallow: /

User-agent: ia_archiver/1.6
Disallow: /

User-agent: Indy\ Library
Disallow: /

User-agent: InfoNaviRobot
Disallow: /

User-agent: ip\-web\-crawler\.com
Disallow: /

User-agent: Iron33/1.0.2
Disallow: /

User-agent: Jakarta\ Commons-HttpClient
Disallow: /

User-agent: Jeeves
Disallow: /

User-agent: JennyBot
Disallow: /

User-agent: Jetbot
Disallow: /

User-agent: Jetbot/1.0
Disallow: /

User-agent: JikeSpider
Disallow: /

User-agent: Kenjin Spider
Disallow: /

User-agent: Keyword Density/0.9
Disallow: /

User-agent: larbin
Disallow: /

User-agent: LexiBot
Disallow: /

User-agent: libWeb/clsHTTP
Disallow: /

User-agent: libwww-perl
Disallow: /

User-agent: lindex\.com
Disallow: /

User-agent: Linguee
Disallow: /

User-agent: linkdex\.com
Disallow: /

User-agent: linkdexbot
Disallow: /

User-agent: LinkextractorPro
Disallow: /

User-agent: LinkScan/8.1a Unix
Disallow: /

User-agent: LinkWalker
Disallow: /

User-agent: lipperhey
Disallow: /

User-agent: LNSpiderguy
Disallow: /

User-agent: looksmart
Disallow: /

User-agent: ltbot
Disallow: /

User-agent: lwp-trivial
Disallow: /

User-agent: lwp-trivial/1.34
Disallow: /

User-agent: Lynx
Disallow: /

User-agent: magpie\-crawler
Disallow: /

User-agent: Mata Hari
Disallow: /

User-agent: Mattermost-Bot
Disallow: /

User-agent: http://MegaIndex.ru
Disallow: /

User-agent: Microsoft URL Control
Disallow: /

User-agent: Microsoft URL Control - 5.01.4511
Disallow: /

User-agent: Microsoft URL Control - 6.00.8169
Disallow: /

User-agent: MIIxpc
Disallow: /

User-agent: MIIxpc/4.2
Disallow: /

User-agent: Mister PiX
Disallow: /

User-agent: MJ12bot
Disallow: /

User-agent: moget
Disallow: /

User-agent: moget/2.1
Disallow: /

User-agent: Mozilla/4.0 (compatible; BullsEye; Windows 95)
Disallow: /

User-agent: Mozilla/5.0 (compatible; Googlebot/2.1; startmebot/1.0; +https://start.me/bot)
Disallow: /

User-agent: MSIE\ or\ Firefox\ mutant
Disallow: /

User-agent: MSIECrawler
Disallow: /

User-agent: naver
Disallow: /

User-agent: NCBot
Disallow: /

User-agent: NetAnts
Disallow: /

User-agent: NetcraftSurveyAgent
Disallow: /

User-agent: netEstate\ NE\ Crawler
Disallow: /

User-agent: Neticle Crawler
Disallow: /

User-agent: NetMechanic
Disallow: /

User-agent: Netseer
Disallow: /

User-agent: NextGenSearchBot
Disallow: /

User-agent: NICErsPRO
Disallow: /

User-agent: Nutch
Disallow: /

User-agent: Ocelli
Disallow: /

User-agent: Offline Explorer
Disallow: /

User-agent: OmniExplorer_Bot
Disallow: /

User-agent: Openbot
Disallow: /

User-agent: Openfind
Disallow: /

User-agent: Openfind data gathere
Disallow: /

User-agent: OpenWebIndex
Disallow: /

User-agent: Oracle Ultra Search
Disallow: /

User-agent: PagesInventory
Disallow: /

User-agent: PEAR
Disallow: /

User-agent: PeoplePal
Disallow: /

User-agent: PerMan
Disallow: /

User-agent: ProCogSEOBot
Disallow: /

User-agent: ProPowerBot/2.14
Disallow: /

User-agent: ProWebWalker
Disallow: /

User-agent: proximic
Disallow: /

User-agent: psbot
Disallow: /

User-agent: PubMatic Crawler Bot
Disallow: /

User-agent: purebot
Disallow: /

User-agent: QueryN Metasearch
Disallow: /

User-agent: QuerySeekerSpider
Disallow: /

User-agent: Radiation Retriever 1.1
Disallow: /

User-agent: RepoMonkey
Disallow: /

User-agent: RepoMonkey Bait & Tackle/v1.01
Disallow: /

User-agent: Riddler
Disallow: /

User-agent: RMA
Disallow: /

User-agent: rojerbot
Disallow: /

User-agent: RyteBot
Disallow: /

User-agent: scooter
Disallow: /

User-agent: ScoutJet
Disallow: /

User-agent: Scrapy
Disallow: /

User-agent: ScreenerBot
Disallow: /

User-agent: searchmetrics
Disallow: /

User-agent: searchpreview
Disallow: /

User-agent: SemrushBot
Disallow: /

User-agent: sentibot
Disallow: /

User-agent: SEO-CRAWLING
Disallow: /

User-agent: SEOENGWorldBot
Disallow: /

User-agent: SEOkicks-Robot
Disallow: /

User-agent: ShopWiki
Disallow: /

User-agent: sistrix
Disallow: /

User-agent: sitebot
Disallow: /

User-agent: SiteSnagger
Disallow: /

User-agent: Snoopy
Disallow: /

User-agent: SocialSearcher
Disallow: /

User-agent: sogou
Disallow: /

User-agent: SolomonoBot
Disallow: /

User-agent: sootle
Disallow: /

User-agent: Sosospider
Disallow: /

User-agent: SpankBot
Disallow: /

User-agent: spanner
Disallow: /

User-agent: spbot
Disallow: /

User-agent: Speedy
Disallow: /

User-agent: Stanford
Disallow: /

User-agent: Stanford Comp Sci
Disallow: /

User-agent: SurveyBot
Disallow: /

User-agent: suzuran
Disallow: /

User-agent: Synapse
Disallow: /

User-agent: Szukacz/1.4
Disallow: /

User-agent: Teleport
Disallow: /

User-agent: TeleportPro
Disallow: /

User-agent: Telesoft
Disallow: /

User-agent: Teoma
Disallow: /

User-agent: The Intraformant
Disallow: /

User-agent: The\ Incutio\ XML-RPC\ PHP\ Library
Disallow: /

User-agent: TheNomad
Disallow: /

User-agent: toCrawl/UrlDispatcher
Disallow: /

User-agent: True_Robot
Disallow: /

User-agent: True_Robot/1.0
Disallow: /

User-agent: turingos
Disallow: /

User-agent: TurnitinBot
Disallow: /

User-agent: uCrawler
Disallow: /

User-agent: URL Control
Disallow: /

User-agent: URL_Spider_Pro
Disallow: /

User-agent: URLy Warning
Disallow: /

User-agent: VCI
Disallow: /

User-agent: VCI WebViewer VCI WebViewer Win32
Disallow: /

User-agent: visaduhoc\.info
Disallow: /

User-agent: WBSearchBot
Disallow: /

User-agent: Web Image Collector
Disallow: /

User-agent: WebAuto
Disallow: /

User-agent: WebBandit
Disallow: /

User-agent: WebBandit/3.50
Disallow: /

User-agent: WebCapture
Disallow: /

User-agent: WebCopier
Disallow: /

User-agent: WebEnhancer
Disallow: /

User-agent: WebInDetail\.com
Disallow: /

User-agent: WebmasterWorld Extractor
Disallow: /

User-agent: WebmasterWorldForumBot
Disallow: /

User-agent: WebSauger
Disallow: /

User-agent: Website Quester
Disallow: /

User-agent: WEBSITEtheWEB\.COM
Disallow: /

User-agent: Webster Pro
Disallow: /

User-agent: WebStripper
Disallow: /

User-agent: WebVac
Disallow: /

User-agent: WebZip
Disallow: /

User-agent: WebZip/4.0
Disallow: /

User-agent: Wget
Disallow: /

User-agent: Wget/1.5.3
Disallow: /

User-agent: Wget/1.6
Disallow: /

User-agent: Wotbot
Disallow: /

User-agent: http://www.trendiction.de/bot
Disallow: /

User-agent: www\.integromedb\.org
Disallow: /

User-agent: WWW-Collector-E
Disallow: /

User-agent: Xenu's
Disallow: /

User-agent: Xenu's Link Sleuth 1.1c
Disallow: /

User-agent: xpymep\.exe
Disallow: /

User-agent: YamanaLab-Robot
Disallow: /

User-agent: Yandex
Disallow: /

User-agent: YisouSpider
Disallow: /

User-agent: YodaoBot
Disallow: /

User-agent: YoudaoBot
Disallow: /

User-agent: Zend_Http_Client
Disallow: /

User-agent: Zeus
Disallow: /

User-agent: Zeus 32297 Webster Pro V2.9 Win32
Disallow: /

User-agent: Zeus Link Scout
Disallow: /

User-agent: ZmEu
Disallow: /

User-agent: ZumBot
Disallow: /

User-agent: SirdataBot (+https://semantic-api.docs.sirdata.net/contextual-api/contextual-api/introduction),"81,530"
Disallow: /

User-agent: Mozilla/5.0 (compatible; Paqlebot/2.0; +http://www.paqle.dk/about/paqlebot),"28,241"
Disallow: /

User-agent: Mediatoolkitbot (complaints@mediatoolkit.com),"24,331"
Disallow: /

User-agent: Mozilla/5.0 (compatible; Semanticbot/1.0; +http://sempi.tech/bot.html),"14,317"
Disallow: /

User-agent: Mozilla/5.0 (compatible; SentiBot/1.0; +https://www.sentibot.eu),"11,025"
Disallow: /

History

Changes recorded since June 2026.

No change recorded.

In context

Poland
36.4% block at least one AI crawler
Behind Cloudflare
No
Same group