CrawlerPulse
CrawlerPulse / Asia / India / dainikgomantak.esakal.com

Dainik Gomantak

dainikgomantak.esakal.com · India · Sakal Media Private Limited · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 624 bytes. No rule names an AI crawler.

User-agent: *
Allow: /
Disallow: /collection/google-news
Disallow: /authorPage/
Disallow: *//*
Disallow: *?utm_*
Disallow: *?amp*
Disallow: /searchPage/
Disallow: *115394472*
Disallow: */api/v1/stories/*
Disallow: */api/auth/v1/users/*
Disallow: */api/v1/breaking-news/*
Disallow: /search*
Disallow: /video-story/
Disallow: *?botrequest=true*
Disallow: */preview/story/*

User-agent: *
Disallow:

User-agent: AdsBot-Google
Disallow:

User-agent: Mediapartners-Google
Disallow:

User-agent: Bingbot
Disallow:
Sitemap: https://dainikgomantak.esakal.com/sitemap.xml
Sitemap: https://dainikgomantak.esakal.com/news_sitemap.xml

History

Changes recorded since June 2026.

No change recorded.

In context

India
25.0% block at least one AI crawler
Behind Cloudflare
Yes
Same group