amarujala.com · India · Amar Ujala Web Services Private Limited · measured 8 Oct 2026
As read on 8 Oct 2026, 2,017 bytes. 20 rule groups name AI crawlers; shown first.
User-agent: CCBot Allow: / User-agent: anthropic-ai Allow: / User-agent: Omgilibot Allow: / User-agent: Omgili Allow: / User-agent: Diffbot Allow: / User-agent: Bytespider Allow: / User-agent: ImagesiftBot Allow: / User-agent: cohere-ai Allow: / User-agent: ClaudeBot Allow: / User-agent: Claude-Web Allow: / User-agent: Meta-ExternalAgent Allow: / User-agent: Applebot-Extended Allow: / User-agent: Amazonbot Allow: / User-agent: GPTBot Allow: / User-agent: Google-Extended Allow: / User-agent: FacebookBot Allow: / User-agent: OAI-SearchBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: PerplexityBot Allow: / User-agent: YouBot Allow: / User-agent: Meta-ExternalFetcher Allow: / User-agent: Applebot Allow: / User-agent: Claude-SearchBot Allow: / User-agent: Meta-WebIndexer Allow: / User-agent: FirecrawlAgent Allow: / User-agent: AndiBot Allow: / User-agent: ExaBot Allow: / User-agent: PhindBot Allow: / User-agent: * Allow: /.well-known/amphtml/apikey.pub Disallow: /blocked.html Disallow: /entertainment/bollywood/poonam-pandey-sami-nude-pics Disallow: /static-ad/home Disallow: /ucv1/ Disallow: /ajax/ Disallow: /cdn-cgi/ Disallow: /sports/parimatch-betting-five-qualities-of-modern-betting-all-you-need-to-know Disallow: /lifestyle/kapiva-himalayan-shilajit-from-18000-feet-himalaya-for-good-physical-and-mental-health Disallow: /tags/betting Disallow: /channels/amp_downloads* Disallow: /tags/sex* Disallow: /games/* Disallow: /real-estate/* Disallow: /advertorial/* Disallow: /form-widget/* Disallow: /search Disallow: /*?q=* Disallow: /*&q=* Disallow: /*?src=* Disallow: /*&src=* Disallow: /*?utm_* Disallow: /*&utm_* Disallow: /*?client* Disallow: /*&client* Disallow: /*?in_app=* Disallow: /*&in_app=* Disallow: /*?position=* Disallow: /*&position=* Disallow: /*?src=ampnext Disallow: /?amp Disallow: /?version Disallow: /?v= Sitemap: https://www.amarujala.com/sitemap.xml Sitemap: https://www.amarujala.com/sitemap-news-v1.xml Sitemap: https://www.amarujala.com/sitemap/sitemap-index.xml
Changes recorded since June 2026.
No change recorded.