CrawlerPulse
CrawlerPulse / Asia / Vietnam / dantri.com.vn

dantri

dantri.com.vn · Vietnam · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 3,410 bytes. 9 rule groups name AI crawlers; shown first.

User-agent: Google-Extended
Crawl-delay: 1

User-agent: PerplexityBot
Crawl-delay: 1

User-agent: OAI-SearchBot
Crawl-delay: 1

User-agent: Applebot-Extended
Crawl-delay: 1

User-agent: GPTBot
Crawl-delay: 1

User-agent: ClaudeBot
Crawl-delay: 1

User-agent: cohere-ai
Crawl-delay: 1

User-agent: Omgilibot
Crawl-delay: 1

User-agent: FacebookBot
Crawl-delay: 1

User-agent: Bingbot
Crawl-delay: 1

User-agent: Googlebot
Crawl-delay: 1

User-agent: bingbot
Crawl-delay: 1

User-agent: facebookexternalhit
Allow: /

User-agent: facebookexternalhit/1.1
Allow: /

User-agent: FacebookExternalHit
Allow: /

User-agent: facebookexternalhit
Disallow:

User-agent: facebookexternalhit/1.1
Disallow:

User-agent: FacebookExternalHit
Disallow:

User-agent: *
Allow: /
Sitemap: https://dantri.com.vn/sitemaps/category-sitemap.xml
Sitemap: https://dantri.com.vn/sitemaps/articles.xml
Sitemap: https://dantri.com.vn/sitemaps/Videos.xml
Sitemap: https://dantri.com.vn/google-news-sitemap.xml
Sitemap: https://dantri.com.vn/sitemaps/collection-sitemap.xml
Disallow: /wp-json/v1/newspaper
Disallow: /print-*.htm
Disallow: /news-*.htm
Disallow: */amp/video/*
Disallow: /*.tag
Disallow: /*/trang-1*.htm
Disallow: /*/trang-2*.htm
Disallow: /*/trang-3*.htm
Disallow: /*/trang-4*.htm
Disallow: /*/trang-5*.htm
Disallow: /*/trang-6*.htm
Disallow: /*/trang-7*.htm
Disallow: /*/trang-8*.htm
Disallow: /*/trang-9*.htm
Disallow: /tim-kiem/*betgg8.net*
Disallow: /wp-json/v1/newspaper
Disallow: /tim-kiem/*✔*
Disallow: /tim-kiem/*▋*
Disallow: /tim-kiem/*💕*
Disallow: /tim-kiem/*⚡*
Disallow: /tim-kiem/*🎖*
Disallow: /tim-kiem/*❤*
Disallow: /tim-kiem/*⚽*
Disallow: /tim-kiem/*@ecnypay*
Disallow: /tim-kiem/*casino666.vip*
Disallow: /tim-kiem/*⭐*
Disallow: /tim-kiem/*tg@macaoai*
Disallow: /tim-kiem/*(*
Disallow: /tim-kiem/*◠*
Disallow: /tim-kiem/*/xodpqbox/*
Disallow: /tim-kiem/*【*
Disallow: /tim-kiem/*鏈*
Disallow: /tim-kiem/*區*
Disallow: /tim-kiem/*[*
Disallow: /tim-kiem/*sbs866.com*
Disallow: /tim-kiem/*jj789.co*
Disallow: /tim-kiem/*🍅*
Disallow: /tim-kiem/*🥕*
Disallow: /tim-kiem/*🍌*
Disallow: /tim-kiem/*🌈*
Disallow: /tim-kiem/*🥑*
Disallow: /tim-kiem/*🍔*
Disallow: /tim-kiem/*🥤*
Disallow: /tim-kiem/*香*
Disallow: /tim-kiem/*港*
Disallow: /tim-kiem/*安心*
Disallow: /tim-kiem/*(@*
Disallow: /tim-kiem/*🥝*
Disallow: /tim-kiem/*👈*
Disallow: /tim-kiem/*入*
Disallow: /tim-kiem/*황*
Disallow: /tim-kiem/*△*
Disallow: /tim-kiem/*지노*
Disallow: /tim-kiem/*★*
Disallow: /tim-kiem/*◀*
Disallow: /tim-kiem/*☏*
Disallow: /tim-kiem/*◁*
Disallow: /tim-kiem/*林*
Disallow: /tim-kiem/*▣*
Disallow: /tim-kiem/*⊙*
Disallow: /tim-kiem/*↖*
Disallow: /tim-kiem/*↘*
Disallow: /tim-kiem/*카*
Disallow: /tim-kiem/*▧*
Disallow: /tim-kiem/*≫*
Disallow: /tim-kiem/*스*
Disallow: /tim-kiem/*이*
Disallow: /tim-kiem/*董*
Disallow: /tim-kiem/*◐*
Disallow: /tim-kiem/*%e2%96%8*
Disallow: /tim-kiem/*clearsettle.vip*
Disallow: /tim-kiem/*▋*
Disallow: /tim-kiem/*◎*
Disallow: /tim-kiem/*↕*
Disallow: /tim-kiem/*金*
Disallow: /tim-kiem/*幣*
Disallow: /tim-kiem/*usdt*
Disallow: /tim-kiem/*%20*
Disallow: /tim-kiem/*.htm?pi=*
Disallow: /*%F0%9F%8E%83*
Disallow: /*人*
Disallow: /*casino*
Disallow: /tim-kiem.htm?s=*

History

Changes recorded since June 2026.

No change recorded.

In context

Vietnam
9.9% block at least one AI crawler
Behind Cloudflare
No