CrawlerPulse
CrawlerPulse / Asia / Cyprus / gundemkibris.com

gundemkibris.com

gundemkibris.com · Cyprus · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 1,582 bytes. 1 rule group name AI crawlers; shown first.

User-agent: GPTBot              # OpenAI (ChatGPT)
User-agent: Google-Extended     # Google (Gemini/Bard)
User-agent: CCBot               # Common Crawl
User-agent: ClaudeBot           # Anthropic (Claude)
User-agent: Applebot-Extended   # Apple Intelligence
User-agent: Omgilibot           # Facebook/Meta AI ve digerleri
User-agent: Diffbot             # Veri cikarma botlari
User-agent: FacebookBot         # Meta AI egitim verileri
User-agent: *
Disallow: /widgets/
Disallow: /widgets/load-widget*
Disallow: /bigdata/*
Disallow: /cdn-cgi/*
Disallow: /index.php
Disallow: /public/
Disallow: /service/*
Disallow: /arama/
Disallow: /api
Disallow: /*?q=
Disallow: /*?preview=
Disallow: /*?utm_source=
Disallow: /*?utm_medium=
Disallow: /*?utm_campaign=
Disallow: /*?page=
Disallow: /*?sayfa=
Disallow: /*&page=
Disallow: /*?sort=
Disallow: /*&sort=
Disallow: /advert/*
Disallow: /admin/
Disallow: /register/
Disallow: /password-reset/
Disallow: /auth/
Disallow: /vidyome/embed/
Disallow: /vendor/*
Disallow: /member/
Disallow: /date
Disallow: /poster
Disallow: /arsiv?date
Disallow: /xpanel
Disallow: /comments/*
Disallow: /themes/*
Disallow: /paylas
Disallow: /font-awesome
Disallow: /.well-known/*
Sitemap: https://www.gundemkibris.com/sitemap.xml
Sitemap: https://www.gundemkibris.com/rss?yandex=turbo

User-agent: facebookexternalhit
Allow: /

User-agent: Facebot
Allow: /

User-agent: Twitterbot
Allow: /

User-agent: LinkedInBot
Allow: /

User-agent: WhatsApp
Allow: /

User-agent: Pinterest
Allow: /

History

Changes recorded since June 2026.

No change recorded.

In context

Cyprus
6.6% block at least one AI crawler
Behind Cloudflare
Yes