arynews.tv · Pakistan · measured 8 Oct 2026
As read on 8 Oct 2026, 2,873 bytes. 17 rule groups name AI crawlers; shown first.
User-agent: ChatGPT-User Allow: / User-agent: OAI-SearchBot Allow: / User-agent: PerplexityBot Allow: / User-agent: Google-Extended Allow: / User-agent: GPTBot Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Claude-Web Disallow: / User-agent: anthropic-ai Disallow: / User-agent: CCBot Disallow: / User-agent: Bytespider Disallow: / User-agent: Amazonbot Disallow: / User-agent: Applebot-Extended Disallow: / User-agent: cohere-ai Disallow: / User-agent: Diffbot Disallow: / User-agent: omgili Disallow: / User-agent: omgilibot Disallow: / User-agent: YouBot Disallow: / Sitemap: https://arynews.tv/sitemap/news_sitemap.xml Sitemap: https://arynews.tv/sitemap_custom_pages.xml Sitemap: https://arynews.tv/sitemap/category_sitemap.xml Sitemap: https://arynews.tv/sitemap_namaz.xml Sitemap: https://arynews.tv/sitemap/live_sitemap.xml User-agent: * Allow: / Disallow: /search Disallow: /search? Disallow: /*?q= Disallow: /*&q= Disallow: /category/off-beat/ Disallow: /off-beat/ Disallow: /category/lifestyle/ Disallow: /lifestyle/ Disallow: /wp-admin/ Disallow: /admin/ Disallow: /author/wp_admin* Disallow: /wp-content/ Disallow: /en/ Disallow: /assets/ Allow: /assets/images/ Allow: /api/author/ Allow: /api/author/* Disallow: /api/ Disallow: /*.json$ Disallow: /ads/preferences/ Disallow: /_next/ Allow: /_next/static/css/ Allow: /_next/static/media/ Allow: /_next/static/chunks/ Disallow: /_next/image Disallow: /*.amp$ Disallow: /tag-*-amp Disallow: /*?count= Disallow: /*&count= Disallow: /author/*?count= Disallow: /author/*&count= User-agent: Perplexity-User Allow: / User-agent: Claude-SearchBot Allow: / User-agent: Claude-User Allow: / User-agent: Scrapy Disallow: / User-agent: magpie-crawler Disallow: / User-agent: HeadlessChrome Disallow: / User-agent: PhantomJS Disallow: / User-agent: Selenium Disallow: / User-agent: Puppeteer Disallow: / User-agent: Playwright Disallow: /
Changes recorded since June 2026.
No change recorded.