gulf-times.com · Qatar · measured 8 Oct 2026
As read on 8 Oct 2026, 890 bytes. No rule names an AI crawler.
User-agent: * Disallow: /ajax/* Disallow: /ajax* Disallow: /print* Disallow: /getRelatedArticles* Disallow: /getMostReadArticles* Disallow: /article_count/* Disallow: /get-menu-header* Disallow: /article.php* Disallow: /login-mgt Disallow: /*.php Disallow: /archive/* Disallow: /rss Disallow: /rssFeed/* Disallow: /widget/* Disallow: */page/* User-agent: Baiduspider Disallow: / User-agent: Baiduspider-video Disallow: / User-agent: Baiduspider-image Disallow: / User-agent: istellabot Disallow: / User-agent: SemrushBot Disallow: / User-agent: DotBot Disallow: / User-agent: bingbot Crawl-delay: 10 User-agent: Yandex Crawl-delay: 10 User-agent: Twitterbot Crawl-delay: 10 User-agent: * Crawl-delay: 2 Sitemap: https://www.gulf-times.com/sitemaps/sitemap_0.xml Sitemap: https://www.gulf-times.com/sitemaps/newsSitemap.xml
Changes recorded since June 2026.
No change recorded.