CrawlerPulse
CrawlerPulse / Asia / Thailand / thethaiger.com

The Thaiger

thethaiger.com · Thailand · measured 9 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Blocked
Amazon
Allowed

robots.txt

As read on 9 Oct 2026, 1,240 bytes. 1 rule group name AI crawlers; shown first.

User-agent: Applebot-Extended
Disallow: /

Sitemap: https://thethaiger.com/sitemap_index.xml
Sitemap: https://thethaiger.com/news-sitemap.xml
Sitemap: https://thethaiger.com/th/sitemap_index.xml
Sitemap: https://thethaiger.com/th/news-sitemap.xml

User-agent: *
Disallow: /wp-admin/
Disallow: /th/wp-admin/
Allow: /wp-admin/admin-ajax.php
Disallow: /staff/
Disallow: /search/
Disallow: /*preview=true
Disallow: /th/*preview=true
Disallow: /*?ref=
Disallow: /*utm_
Disallow: /*fbclid
Disallow: /*gclid
Disallow: /*gsid=
Disallow: /*sm_guid=
Disallow: /*buffer_share=
Disallow: /*fb_action_ids=
Disallow: /*?hl=
Disallow: /*?platform=hootsuite
Disallow: /*?url=
Disallow: /*?to=
Disallow: /*replytocom

History

Changes recorded since June 2026.

No change recorded.

In context

Thailand
10.7% block at least one AI crawler
Behind Cloudflare
Yes