CrawlerPulse
CrawlerPulse / Asia / India / naidunia.com

naidunia.com

naidunia.com · India · Dainik Jagran · measured 8 Oct 2026

Google Search
Allowed
Google-Extended
Allowed
OpenAI
Allowed
Anthropic
Allowed
Perplexity
Allowed
Meta
Allowed
Common Crawl
Allowed
ByteDance
Allowed
Apple
Allowed
Amazon
Allowed

robots.txt

As read on 8 Oct 2026, 681 bytes. No rule names an AI crawler.

User-agent: *
Allow: /
Disallow: /*.aspx*
Disallow: /*.asp*
Disallow: /*php?id*
Disallow: /*13276288*
Disallow: /home
Disallow: */search/*
Disallow: *?itm*
Disallow: *-page1*
Disallow: *-page2*
Disallow: *-page3*
Disallow: *-page4*
Disallow: *-page5*
Disallow: *-page6*
Disallow: *-page7*
Disallow: *-page8*
Disallow: *-page9*
Disallow: */photo-gallery/*
Disallow: */topics-trending/*
Disallow: */trending/*
Disallow: /pfp
Disallow: /pfp/
Disallow: /pfp/*
Disallow: *?utm_source*
Disallow: *?utm_medium*
Disallow: /Resources/mnaidunia/plt/
Disallow: */cricket11/*
Sitemap: https://www.naidunia.com/news-sitemap-nd.xml
Sitemap: https://www.naidunia.com/web-sitemap-nd.xml

History

Changes recorded since June 2026.

No change recorded.

In context

India
25.0% block at least one AI crawler
Behind Cloudflare
No
Same group