CrawlerPulse
CrawlerPulse / North America / United States / kulr8.com

NonStop Local

kulr8.com · United States · Cowles Company · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Blocked
OpenAI
Blocked
Anthropic
Blocked
Perplexity
Blocked
Meta
Blocked
Common Crawl
Blocked
ByteDance
Blocked
Apple
Blocked
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 4,009 bytes. 19 rule groups name AI crawlers; shown first.

User-agent: ChatGPT-User
Disallow: /

User-agent: Bytespider
Disallow: /

User-agent: CCBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: Diffbot
Disallow: /

User-agent: FacebookBot
Disallow: /

User-agent: Google-Extended
Disallow: /

User-agent: GPTBot
Disallow: /

User-agent: omgili
Disallow: /

User-agent: Applebot-Extended
Disallow: /

User-agent: anthropic-ai
Disallow: /

User-agent: ImagesiftBot
Disallow: /

User-agent: Meta-ExternalAgent
Disallow: /

User-agent: PerplexityBot
Disallow: /

User-agent: PetalBot
Disallow: /

User-agent: Timpibot
Disallow: /

User-agent: YouBot
Disallow: /

User-agent: cohere-ai
Disallow: /

User-agent: omgilibot
Disallow: /

User-agent: AI2Bot
Disallow: /

User-agent: AI2Bot-Dolma
Disallow: /

User-agent: FriendlyCrawler
Disallow: /

User-agent: ICC-Crawler
Disallow: /

User-agent: Scrapy
Disallow: /

User-agent: VelenPublicWebCrawler
Disallow: /

User-agent: Webzio-Extended
Disallow: /

User-agent: iaskspider/2.0
Disallow: /

User-agent: img2dataset
Disallow: /

User-agent: *
Disallow: /tncms/tracking/
Disallow: /_services/
Disallow: /contests/2026_holiday_bucks_giveaway/
Disallow: /contests/2026_riggin_masters_ticket_giveaway/
Disallow: /contests/backyards-bbq/
Disallow: /contests/brews_bbqs/
Disallow: /contests/gold_star_award/
Disallow: /contests/holiday_nights_tent_giveaway/
Disallow: /contests/kulr_the_weather/
Disallow: /contests/two_men_and_a_truck_3_for_free_2026/
Disallow: /olympics/
Disallow: /olympics/nbc/
Disallow: /test/
Disallow: /test/nxt/
Disallow: /tncms/openid2/
Disallow: /tncms/weather/
Disallow: /tncms/admin/
Disallow: /tncms/messaging/
Disallow: /tncms/webservice/
Disallow: /tncms/gtm/
Disallow: /tncms/counter/
Disallow: /tncms/dmp/
Disallow: /tncms/track/
Disallow: /tncms/media/
Disallow: /tncms/csrf/
Disallow: /tncms/tax/
Disallow: /tncms/block/
Disallow: /tncms/search/
Disallow: /tncms/openweb/*
Disallow: /tncms/auth/
Disallow: /tncms/calendar/
Disallow: /calendar/search/
Disallow: /calendar/arts/search/
Disallow: /calendar/comedy/search/
Disallow: /calendar/community/search/
Disallow: /calendar/community_events/search/
Disallow: /calendar/educational/search/
Disallow: /calendar/family/search/
Disallow: /calendar/family_friendly/search/
Disallow: /calendar/festivals/search/
Disallow: /calendar/food/search/
Disallow: /calendar/holiday/search/
Disallow: /calendar/meetings/search/
Disallow: /calendar/music/search/
Disallow: /calendar/music_and_arts/search/
Disallow: /calendar/sports/search/
Disallow: /calendar/sports_recreation/search/
Disallow: /tncms/user/
Disallow: /users/admin/
Disallow: /users/*/?
Disallow: /content/tncms/
Disallow: /tncms/disqus/
Disallow: /content/tncms/assets/v3/form/
Disallow: /content/tncms/form/
Disallow: /marketplace/*action=srch
Sitemap: https://www.kulr8.com/sitemap.xml
Sitemap: https://www.kulr8.com/tncms/sitemap/editorial.xml
Sitemap: https://www.kulr8.com/tncms/sitemap/editorial.xml?year=2012
Sitemap: https://www.kulr8.com/tncms/sitemap/editorial.xml?year=2013
Sitemap: https://www.kulr8.com/tncms/sitemap/editorial.xml?year=2014
Sitemap: https://www.kulr8.com/tncms/sitemap/editorial.xml?year=2015
Sitemap: https://www.kulr8.com/tncms/sitemap/editorial.xml?year=2016
Sitemap: https://www.kulr8.com/tncms/sitemap/editorial.xml?year=2017
Sitemap: https://www.kulr8.com/tncms/sitemap/editorial.xml?year=2018
Sitemap: https://www.kulr8.com/tncms/sitemap/editorial.xml?year=2019
Sitemap: https://www.kulr8.com/tncms/sitemap/editorial.xml?year=2020
Sitemap: https://www.kulr8.com/tncms/sitemap/editorial.xml?year=2021
Sitemap: https://www.kulr8.com/tncms/sitemap/editorial.xml?year=2022
Sitemap: https://www.kulr8.com/tncms/sitemap/editorial.xml?year=2023
Sitemap: https://www.kulr8.com/tncms/sitemap/editorial.xml?year=2024
Sitemap: https://www.kulr8.com/tncms/sitemap/editorial.xml?year=2025

History

Changes recorded since June 2026.

No change recorded.

In context

United States
63.2% block at least one AI crawler
Behind Cloudflare
No
Same group