newsyemen.net · Yemen · measured 8 Oct 2026
As read on 8 Oct 2026, 2,461 bytes. 7 rule groups name AI crawlers; shown first.
User-agent: GPTBot Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: CCBot Disallow: / User-agent: anthropic-ai Disallow: / User-agent: Claude-Web Disallow: / User-agent: Omgilibot Disallow: / User-agent: FacebookBot Disallow: / Sitemap: https://www.newsyemen.net/sitemap.xml Sitemap: https://www.newsyemen.net/news-sitemap.xml User-agent: * Disallow: /admin/ Disallow: /core/ Disallow: /config/ Disallow: /migrations/ Disallow: /cron/ Disallow: /logs/ Disallow: /backups/ Disallow: /api/ Disallow: /cache/ Disallow: /search? Disallow: /search/ Disallow: /404.php Disallow: /contact_handler.php Allow: / Allow: /news*.html Allow: /cat*.html Allow: /content.php Allow: /wiki/ Allow: /opinions/ Allow: /feed/ Allow: /assets/ Allow: /user_images/ User-agent: Googlebot-Image Allow: /user_images/ Allow: /assets/images/ User-agent: Googlebot-News Allow: / Disallow: /admin/ Disallow: /core/ Disallow: /config/
Changes recorded since June 2026.
No change recorded.