CrawlerPulse
CrawlerPulse / Asia / India / assamtribune.com

assamtribune.com

assamtribune.com · India · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 1,758 bytes. No rule names an AI crawler.

User-agent: *
Allow: /
Disallow: /admin/*
Disallow: /search/*
Disallow: /search?*
Disallow: /xhr/*
Disallow: /preview/story-*
Disallow: /amp/preview/story-*
Disallow: /staging/*
Disallow: /alfoo
Disallow: /sildoo
Disallow: /dutas
Disallow: /gsearch?*
Disallow: /video-only-for-doctors-72491
Disallow: /introducing-medical-dialogues-medical-and-health-news-portal-74410
Disallow: /epaper/Planet_Young/main.html
Disallow: /epaper/Planet_Young/main.html
Disallow: https://dainikasam.assamtribune.com/amp/aiming-for-the-sky-4/?type=xhr&startIndex=12
Disallow: https://dainikasam.assamtribune.com/amp/aiming-for-the-sky-4/?type=xhr&startIndex=12
Disallow: https://dainikasam.assamtribune.com/movie-watch-4/
Disallow: https://dainikasam.assamtribune.com/jocoserious-4/
Disallow: https://dainikasam.assamtribune.com/aiming-for-the-sky-4/
Disallow: https://epaper.assamtribune.com/i-me-myself-4/
Disallow: https://epaper.assamtribune.com/feelings-5/
Disallow: https://dainikasam.assamtribune.com/stem-corner-4/
Disallow: https://dainikasam.assamtribune.com/indian-history-for-upsc-apsc-exam-4/
Disallow: https://dainikasam.assamtribune.com/aiming-for-the-sky-5/
Disallow: https://dainikasam.assamtribune.com/indian-history-for-upsc-apsc-exam-5/
Disallow: https://dainikasam.assamtribune.com/subscription
Disallow: /epaper/pdf/2021/02/01/the-assam-tribune/1864
Disallow: https://epaper.assamtribune.com/poem-of-the-week-5/
Disallow: https://dainikasam.assamtribune.com/culinary-corner-4/
Disallow: /metsmall
Allow: /xhr/getNewsMixin*
Allow: /content/servlet/RDESController?*
Sitemap: https://assamtribune.com/news-sitemap-daily.xml
Sitemap: https://assamtribune.com/feeds.xml

History

Changes recorded since June 2026.

No change recorded.

In context

India
25.0% block at least one AI crawler
Behind Cloudflare
Yes